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EXECUTIVE  SUMMARY 


Emergency  preparedness  has  changed  with  the  times.  It  used  to  be  that  disasters  were 
viewed  as  independent  events  with  scenario-specific  planning,  response  and  recovery 
issues.!  After  September  11,  terrorism  became  the  focus  of  preparedness  efforts,  and 
later,  following  Hurricane  Katrina  in  2005,  the  focus  shifted  again  to  an  all-hazards 

approach.  2 

Regardless  of  the  nature  of  the  emergency,  collaboration  among  competing 
agencies  is  often  a  difficult  proposal.  Each  agency  has  individualized  needs,  and  its 
survival  may  depend  upon  satisfying  these  needs  before  fulfilling  those  outside  its  area  of 
influence.  In  order  to  prepare  a  community  to  respond  to  and  recover  from  any  disaster,  it 
is  important  that  the  community  views  itself  as  a  whole. 

To  define  a  community  may  be  difficult.  There  are  groups  within  each  agency. 
Each  municipal  agency  or  private  stakeholder  is  part  of  the  greater  community,  and 
therefore,  has  a  responsibility  to  participate  in  its  collaborative  preparedness.  Eire  and 
police  departments  may  view  themselves  as  independent  groups,  but  they  will  work 
together  with  emergency  medical  personnel  as  members  of  a  first  responder  collective. 
Agencies,  public  and  private,  may  consider  themselves  part  of  a  community  that  is  not 
even  aware  of  them,  as  may  be  the  case  with  private  security  and  local  law  enforcement.  3 
The  members  of  each  group  must  identify  themselves  as  members  of  the  community  first 
and  as  members  of  individual  factions  second.  In  this  way,  they  can  contribute  to  the 
improvements  of  the  collective  and  all  groups  reap  the  benefits. 

Agencies  conduct  preparedness  exercises  in  an  attempt  to  ready  their  personnel  to 
respond  appropriately  in  the  event  of  an  emergency.  These  exercises  are  evaluated  and 
analyzed,  and  the  results  are  provided  to  the  agency  in  an  effort  to  correct  areas  for 

!  Donald  A.  Donahue  et  al.,  “The  All  Needs  Approach  to  Emergency  Response,”  Homeland  Security 
Affairs  8,  no.  1  (February  2012). 

2  Department  of  Homeland  Security,  Quadrennial  Homeland  Security  Review  Report  (New  York: 
Department  of  Homeland  Security,  February  2010). 

3  John  Collicutt,  “Community  Resilience:  The  Future  of  Business  Continuity,”  Journal  of  Business 
Continuity  &  Emergency  Planning  3,  no.  2  (2009):  145-152. 
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improvement  or  sustain  strengths.  The  purpose  of  exereise  reeommendations  is  to  foster 
improvements.  The  same  ean  be  said  for  reeommendations  from  real-world  events.  Issues 
identified  during  these  should  be  incorporated  into  future  exercises  and  into  an  agency’s 
corrective  actions. 

In  order  to  create  a  baseline  to  demonstrate  the  relationship  between 
recommendations  and  exercise  objectives,  a  study  of  After  Action  Reports  (AAR)  was 
conducted.  Designed  to  compare  and  contrast  recommendations  and  objectives  from 
exercises  and  real-world  emergencies,  this  study  revealed  a  series  of  inconsistencies. 
Several  recommendations  appeared  repeatedly  and  received  minimal  inclusion  into 
exercise  planning.  This  gap  results  in  exercises  not  achieving  their  desired  effect.  Until 
the  coordination  between  recommendation  and  objective  is  realized,  exercise  planners 
may  continue  to  miss  their  mark,  and  exercises  will  fail  to  deliver  the  desired  impact  on 
emergency  preparedness. 

The  intelligence  process  is  cyclically  based.  Information  is  collected,  analyzed 
then  disseminated.  This  affects  the  consumer’s  planning  and  needs  and  the  result  is 
evaluated.  The  feedback  provided  to  the  producers  of  intelligence  influence  the 
information  that  is  collected  and  the  cycle  continues.  An  exercise  design  program  is 
likewise  cyclical.  Exercises  are  designed  and  conducted  based  on  an  agency’s  needs.  The 
resultant  actions  are  evaluated  and  incorporated  into  planning  for  systematic 
improvement  and  future  exercises.  Corrective  actions  are  implemented  and  reinserted 
into  the  design  of  future  exercises. 

A  strengthening  of  the  exercise  process  can  be  anticipated  if  the  two  cycles 
intersect  at  the  planning  phase  and  intelligence  is  inserted  into  the  exercise  cycle. 
Exercise  would  then  provide  feedback  to  the  intelligence  analysts  providing  them  with 
findings  based  on  evaluation.  This  information  would  allow  the  intelligence  professionals 
to  both  look  for  information  relevant  for  future  exercises  and  analyze  trends  revealed 
during  exercises  that  may  support  future  intelligence  paths. 


Keeley  Townsend  et  al.,  “Intelligence-Led  Mitigation,”  Journal  of  Homeland  Security  and 
Emergency  Management  1 ,  no.  1  (2010). 
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The  fusion  of  exercise  design  with  intelligence  creates  a  supportive  relationship 
between  both  that  improves  the  abilities  of  each.  As  each  profession  works  toward  its 
own  goals  and  objectives  with  a  common  focus  toward  the  needs  of  others  as  well,  the 
entire  process  begins  to  produce  a  more  effective  product.  This  gives  birth  to  the  idea  of 
Intelligent  Exercise  Theory  (lET)  and  The  Intelligent  Exercise  Model  (lEM).  As  agencies 
embrace  the  lEM  and  begin  to  view  preparedness  exercises  as  a  natural  outreach  of  the 
intelligence  process,  it  can  be  expected  that  communications  between  the  intelligence  and 
exercise  communities  will  increase.  Increased  communications  lead  to  increased  patterns 
of  interaction,  coordination  and  decision-making.  ^  Since  a  disaster  is  ultimately  measured 
by  a  community’s  ability  to  withstand  its  impact,  this  increase  in  coordinated  abilities 
should  strengthen  the  community’s  resolve  when  faced  with  the  future  adversity.^ 

Ultimately,  as  a  result  the  development  of  the  lEM  and  adoption  of  Intelligent 
Exercise,  the  agencies  of  New  York  City  can  build  a  cooperative  exercise  design 
community  made  up  of  public  and  private  stakeholders  and  supported  by  the 
municipality.  Creating  a  platform  where  entities  can  benefit  freely  from  each  other’s 
exercises  and  intelligence  analysis  will  encourage  interagency  collaboration  and  a 
common-goals  approach  to  NYC  preparedness. 


^  Clayton  M.  Christensen,  The  Innovator’s  Dilemma:  When  New  Technologies  Cause  Great  Firms  to 
Fail  (Management  of  Innovation  and  Change),  Reprint  edition  (Boston:  Harvard  Business  Review  Press, 
2013),  208. 

^  C.  Lomnitz,  “What  Is  a  Disaster?”  Natural  Hazards  18,  no.  1  (1998):  87-88. 
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I.  INTRODUCTION 


In  the  midst  of  chaos,  there  is  also  opportunity. 

— Sun  Tzu,  The  Art  of  War 


A,  INTRODUCTION 

A  population  will  be  calmer  in  the  aftermath  of  a  terrorist  attack  or  natural 
disaster  if  they  know  what  to  expect.  ^  The  municipality  has  a  responsibility  to  train  its 
employees  on  how  to  respond  in  the  event  of  such  a  disaster.  Through  a  robust  exercise 
program,  agencies  can  better  prepare.  Since  2004,  the  Homeland  Security  Exercise 
Evaluation  Program  (HSEEP)  has  offered  guidance  to  agencies  to  conduct  and  develop 
exercises  based  on  capabilities  to  prepare  first  responders  and  citizens  for  disasters, 
natural  and  man-made  based  on  an  area’s  risk/vulnerability  analysis. 2  By  injecting 
intelligence  into  the  risk/vulnerability  analysis,  a  more  refined  and  effective  exercise 
program  can  be  maintained  contributing  the  greater  improvement  of  an  area’s  community 
resilience. 

B,  PROBLEM  STATEMENT 

The  Eire  Department  of  the  City  of  New  York’s  (EDNY)  Center  for  Terrorism 
and  Disaster  Preparedness  (CTDP)  operates  a  Eire  Service  Intelligence  branch  as  well  as 
an  Exercise  Design  program.  These  two  entities  provide  the  EDNY  and  other  public  and 
private  city  agencies  with  exceptional  levels  of  exercise  options,  as  well  as  critically 
astute  analytical  intelligence  from  a  fire  service  perspective.  The  intelligence  section 
weighs  information  from  around  the  world,  from  both  classified  and  open  sources,  and 
provides  relevant  analysis  to  first  responders.  At  the  same  time,  the  exercise  design 
branch  partners  with  various  agencies  throughout  the  city  to  create  realistic  exercise 
scenarios  that  serve  the  needs  of  the  department  as  well  as  those  of  cooperating  agencies. 

^  Consuela  B.  Pocket!,  United  States  and  Israeli  Homeland  Security:  A  Comparative  Analysis  of 
Emergency  Preparedness  Efforts,  The  Counterproliferation  Papers  Future  Warfare  Series  (Maxwell  Air 
Force  Base,  AL:  Counterproliferation  Center,  August  2005). 

2  “HSEEP-LLIS,”  Department  of  Homeland  Security,  accessed  February  6,  2014, 
https://www.llis.dhs.gOv/.2004. 
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The  cooperative  fusion  of  these  two  branches  exponentially  increases  the  capabilities  of 
each,  resulting  in  a  progressive,  intelligence-led  exercise  design  program.  Ideally, 
intelligence  should  provide  the  exercise  design  team  with  relevant  analysis  of  current 
threats  facing  city  properties  and  personnel.  This,  in  turn,  should  enable  the  exercise 
design  team  to  develop  practical  scenarios  based  on  realistic  concerns. 

Unfortunately,  intelligence  and  exercise  professionals  often  operate  independently 
of  each  other  and  often  do  not  offer  cooperative  solutions  to  their  department’s  needs. 
The  problem  lies,  not  simply  with  a  lack  of  coordination  between  intelligence  and 
exercise  design,  but  in  the  inability  to  overcome,  what  the  9/11  Commission  referred  to  as 
“the  human  or  systemic  resistance  to  sharing  information.  ”3  This  reluctance  is  not  unique 
within  the  Department  and  can  be  seen  on  every  level  within  the  intelligence 
community. 4  To  its  credit,  the  CTDP  intel  section  collaborates  with  the  intelligence 
branches  of  other  agencies,  including  the  United  States  Coast  Guard  (USCG),  Nassau 
County  and  NYC’s  Office  of  Emergency  Management  (OEM),  and  the  Nationwide 
Suspicious  Activity  Reporting  (SAR)  Initiative  with  the  New  York  State  Intelligence 
Center  (NYSIC).  Internally,  the  EDNY’s  Bureau  of  Eire  Investigation  (BEI)  is  another 
partner.  Yet  there  is  a  noticeable  disconnect  between  two  internal  sections  within  the 
CTDP  itself.  In  order  to  jointly  fulfill  the  obligation  to  satisfy  the  needs  of  the  department 
this  gap  needs  to  be  addressed  and  minimized  through  changes  to  policy  and  culture. 

It  should  be  noted  that  there  is  a  systemic  disconnect  between  the  exercise  design 
and  intelligence  communities.  Eimiting  the  involvement  of  intelligence  in  the  design  of 
exercise  limits  the  breadth  of  the  exercise  and  thus  minimizes  the  lessons  learned.  The 
fire  service  historically  uses  intelligence  to  fight  fires,  especially  those  likely  to  occur 
again.  Intelligence  gathered  today  should  also  help  mitigate  the  challenges  of  tomorrow. 

Exercise  design  is  often  based  on  the  narrowly  focused  needs  of  individual 
stakeholders  and  facilities  rather  than  on  the  more  global  needs  of  the  community.  When 

3  National  Commission  on  Terrorist  Attacks  upon  the  United  States,  The  9/11  Commission  Report: 
Final  Report  of  the  National  Commission  On  Terrorist  Attacks  Upon  the  United  States  (Washington,  DC: 
National  Commission  on  Terrorist  Attacks  upon  the  United  States,  2004). 

^  Richard  A.  Best  Jr,  Intelligence  Information:  Need-to-Know  vs.  Need-to-Share  (Darby,  PA:  Diane 
Publishing,  2011). 
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agencies  conduct  parallel  exercises  separately,  rather  than  exercising  together,  the 
opportunity  to  benefit  from  each  other’s  policies,  procedures  and  lessons  is  lost.  Since 
events  and  emergencies  will  undoubtedly  involve  multiple  agencies,  exercises  should  be 
conducted  using  as  many  agencies  as  appropriate.  Objectives  for  these  exercises  must 
support  the  needs  of  each  agency  based  on  current  intelligence  and  recommendations 
from  previous  events.  The  resultant  chaos  created  by  this  collaborative  gap,  and  the 
disconnect  that  exists  between  intelligence  and  exercise,  creates  an  opportunity  to 
strengthen  the  capabilities  of  each  section. 

The  development  of  a  model  that  will  regularly  incorporate  intelligence  into 
exercise  design  planning  will  allow  for  consistent  reevaluation  of  an  agency’s  needs  and 
capabilities  weighed  against  current  vulnerabilities  and  threats.  If  intelligence  is  re¬ 
circulated  into  after  action  reports,  then  a  steady  succession  of  improvement  can  be 
developed.  This  model,  then  applied  to  an  agency’s  intelligence  and  exercise  design 
sections,  will  serve  as  a  guideline  to  integrate  the  two  and  enable  a  consistent  stream  of 
intelligence-led  exercise  design  and  intelligence  heavy  After  Action  Reports  (AARs), 
culminating  in  recommendations  based  on  experience. 

A  2010  Journal  of  Homeland  Security  and  Emergency  Management  article 
explained  the  progression  from  Intelligence-led  Policing  to  Intelligence-led  Mitigation. 
Intelligent  exercise  is  based  on  an  extrapolation  of  Intelligence-led  Mitigation.  ^ 

The  premise  of  intelligence-led  mitigation  includes  the  intelligence-led 
policing  approach  for  systematically  collecting,  organizing,  analyzing,  and 
utilizing  intelligence  to  make  informed  resource  decisions.  However, 
while  the  goal  of  intelligence-led  policing  is  to  support  law  enforcement 
actions  for  crime  prevention,  disruption,  and  investigation,  the  goal  of 
intelligence-led  mitigation  is  to  provide  organizations  with  public  safety 
and  emergency  service  functions  (including  law  enforcement)  with 
intelligence  products  which  enhance  their  understanding  of  the  operational 
environment  and  enable  them  to  make  informed  resource  decisions  on 
appropriate  preparedness,  prevention,  protection,  response,  and  recovery 
actions  to  mitigate  incidents.^ 


^  Keeley  Townsend  et  al.,  “Intelligence-Led  Mitigation,”  Journal  of  Homeland  Security  and 
Emergency  Management  1 ,  no.  1  (2010). 

6  Ibid. 
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C.  THESIS  EXPECTATIONS 

The  Intelligent  Exercise  Model  (lEM)  was  created  to  demonstrate  the  cyclic 
benefits  that  can  be  developed  when  intelligence  and  exercise  communities  support  each 
other.  This  thesis  will  explain  the  fusion  of  exercise  and  intelligence  industries  and  how 
their  collaboration  will  benefit  both.  Eike  intelligence-led  mitigation,  the  idea  of  an 
intelligence-led  exercise  is  to  think  proactively  and  use  intelligence  to  enhance 
preparedness  and  minimize  the  impact  of  future  disasters. 

The  thesis  will  explore  issues  of  organizational  learning  and  seek  to  understand  a 
lessons-leamed  approach  to  improvement,  despite  agency  biases  and  focus  on  the 
principles  of  motivation  and  human  decision  making  during  disasters.^  As  groups  expand 
and  view  themselves  as  part  of  a  larger  group,  e.g.,  the  community,  they  are  less 
organizationally  protective  than  when  they  view  themselves  as  only  part  of  a  much 
smaller  group,  e.g.,  an  individual  agency.  It  will  analyze  a  broad  representation  of  the 
intelligence  and  exercise  communities  affecting  Homeland  Security  primarily  associated 
with  local  municipalities.  The  thesis  will  then  offer  recommendations  to  fuse  the  interests 
of  both  the  exercise  design  and  intelligence  communities  on  a  broad  scale.  By  presenting 
a  mutually  beneficial  proposal,  it  is  believed  that  Intelligent  Exercise  Theory  will  show 
how  the  intelligence  and  exercise  cycles  can  operate  in  synchronicity. 

The  purpose  of  this  thesis  is  to  address  the  greater  intelligence  and  exercise  design 
communities  and  expose  an  underlying  inability  to  learn  from  one  another  and  envision 
methods  for  operational  collaboration  between  exercise  and  intelligence  to  operate  in 
smarter  collaboration.  After  Action  Reports  (AARs)  from  exercises  and  real-world  events 
are  being  largely  ignored  as  potential  objectives  in  future  exercises.  This  suggests  that  the 
emergency  preparedness  community  is  content  to  maintain  the  status  quo  rather  than 
aggressively  seek  to  improve  community  resilience.  By  developing  and  presenting  the 
Intelligent  Exercise  Model  (lEM)  and  Intelligent  Exercise  Theory  (lET),  this  thesis  will 
offer  more  positive  and  effective  exercise  design  options. 


^  Donald  A.  Donahue  et  al.,  “The  All  Needs  Approach  to  Emergency  Response,”  Homeland  Security 
Affairs  8,  no.  1  (February  2012). 
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II.  LITERATURE  REVIEW 


A.  EXERCISE  DESIGN 

Exercise  can  serve  functions  including  training,  gap  analysis,  planning,  and  team 
building.  8  As  part  of  improvement  planning,  preparedness  exercises  can  be  used  to 
identify  areas  where  improvement  is  needed;  to  inform  the  process  of  improvements  to 
the  plans  and  procedures  to  enhance  performance;  and  finally  to  re-exercise  what 
evaluation  has  yielded  as  issues.  \n  Emergency  Management  Exercises,'^  Regina  Phelps,  a 
consultant  in  the  field  of  exercise  since  1982,  discusses  exercises  as  a  key  to  improving 
communications  among  disciplines.  The  book  explores  the  planning  of  exercises  and 
identifies  the  benefits  of  a  sound  exercise  program.  Exercises  are  conducted  in  order  to 
validate  training,  “and  practice  strategic  and  tactical  prevention,  protection,  response,  and 
recovery  capabilities  in  a  risk-reduced  environment.” 

Much  of  the  exercise  literature  was  gathered  from  over  100  After  Action  Reports 
(AARs),  the  AAR  being  the  key  evaluation  deliverable,  along  with  the  accompanying 
improvement  plan.  The  AAR  should  be  viewed  as  both  a  science  and  an  art  ^  and  should 
not  be  taken  as  insignificant.  After-action  reviews  identify  how  to  correct  deficiencies, 
maintain  the  sustainability  of  our  strengths,  and  focus  on  performance  of  specific  training 
objectives.  According  to  a  May  2013  training  document  from  the  United  States  Eire 
Administration  (USEA),  AARs  should  answer  the  following  5  questions: 

•  What  was  our  mission?  (Did  we  plan  for  this  event?  Were  there  any  gaps 
in  our  planning?) 

•  What  went  well?  (Did  we  have  adequate  resources?  Did  we  do  all  that  we 
could?) 


^  Lisa  R.  Shugarman  et  af ,  Enhancing  Public  Health  Preparedness  (Santa  Monica,  CA:  RAND, 
2005). 

^  Regina  Phelps,  Emergency  Management  Exercises:  From  Response  to  Recovery:  Everything  You 
Need  to  Know  to  Design  a  Great  Exercise  (San  Francisco,  CA:  Chandi  Media,  2011). 

“About  the  Exercise  &  Training  Program,”  The  Kansas  Department  of  Health  and  Environment, 
accessed  February  5,  2014,  http://www.kdhe-exercises.org/. 

1 1  Margaret  S.  Salter  and  Gerald  E.  Klein,  After  Action  Reviews:  Current  Observations  and 
Recommendations  (Arlington,  VA:  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences, 
2007),  http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA463410. 
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•  What  could  have  gone  better?  (Did  we  observe  any  unsafe  behaviors?  Was 
our  training  effeetive?) 

•  What  might  we  have  done  differently?  (If  we  ran  the  same  ineident,  what 
would  we  do  differently?) 

•  Who  needs  to  know?  (What  needs  to  be  fixed,  and  how  do  we  pass  this 
information  to  the  eorreet  source?) 

Several  artieles  in  the  Journal  of  Business  Continuity  &  Emergency  Planning 
explore  exereise  potential  both  in  the  areas  of  development  and  evaluation.  In  order  to 
build  a  tool  that  eneourages  doeumentation  and  analysis  of  relevant  observations  in 
exereises  or  real-world  ineidents  to  support  deeision  makers  in  real  time  the  evaluation 
proeess  needs  to  be  streamlined.  This  would  enable  the  evaluators  to  faeilitate  the  sharing 
of  lessons  learned,  best  praetiees  and  strategie  improvements  for  the  handling  of  future 
disasters  and  improve  a  “tool  for  eolleeting,  proeessing  and  disseminating  disaster-related 
information,  through  its  integration  in  a  series  of  exereises.”  These  exereises,  whieh  ean 
be  eondueted  to  benefit  multiple  ageneies  as  suggested  by  Margaret  Criehton  and  Terenee 
Kelly,  need  to  be  developed  through  a  three-stage  proeess  of  identifying  the  requirement 
for  an  exereise,  obtaining  approval  for  operations,  and  eondueting  an  after-aotion  review 
as  part  of  a  eontinuous  improvement  eyele.^^  While  Crichton  and  Kelly  address 
radioaetive  material  transportation  events,  many  of  the  situational  faetors  ean  be 
generalized  for  the  design  of  exereises  in  other  areas  with  hazardous  eoneems. 

The  uniqueness  of  health  eare-related  exereises  is  explored  in  Making  Acute 
Hospital  Exercises  More  Realistic  Without  Impacting  on  Healthcare  Deliver.  Many  times 
exereises  are  eondueted  in  areas  where  serviee  to  the  publie  eannot  be  interrupted,  and 


“AAP:  After  Action  Review  in  the  Fire  Service,”  United  States  Fire  Administration,  accessed 
October  28,  2013,  https://www.usfa.fema.gov/downloads/pdf/coffee-break/ms/ms_2013_3.pdf). 

12  Stewart  D.  Smith  et  at,  “Disaster  and  Exercise  Performance  Information  Collection  Tool:  Capturing 
Observations  in  Four  Minutes  or  Less,”  Journal  of  Business  Continuity  &  Emergency  Planning  6,  no.  2 
(2013):  151-163. 

I'l  Margaret  Crichton  and  Terence  Kelly,  “Developing  Emergency  Exercises  for  Hazardous  Material 
Transportation:  Process,  Documents  and  Templates,”  Journal  of  Business  Continuity  &  Emergency 
Planning^,  no.  1  (2012):  32-46. 
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the  benefits  of  exercise  need  to  be  carefully  balanced  with  the  organization’s 
responsibilities. 


1.  Scenario  Development 

The  inherent  strategy  of  exercise  design  is  to  develop  an  objectives-based 
scenario  that  allows  the  participants  to  validate  their  training  in  a  realistic  environment.  It 
is  important  to  consider  the  effect  of  scenario  planning  on  exercise  design.  Scenario 
development  gained  popularity  following  World  War  II  and  was  later  used  by 
corporations  to  speculate  on  economics,  society  and  environmental  concerns,  While  the 
term  “scenario  planning”  does  not  mean  the  same  thing  when  applied  to  different 
strategies  or  organizations,!^  an  acceptable  definition  of  scenarios  would  include  that 
they  “are  consistent  and  coherent  descriptions  of  alternative  hypothetical  futures  that 
reflect  different  perspectives  on  past,  present,  and  future  developments,  which  can  serve 
as  a  basis  for  action. 

Scenario  planning  is  not  a  prediction,  despite  the  various  meanings  of  the  term.i^ 
It  is  intended  to  help  the  strategizing  organization  determine  whether  they  are  adequately 
equipped  to  handle  future  challenges. 20  As  we  study  the  inclusion  of  scenario  planning,  it 
is  important  to  look  beyond  one’s  own  industry  and  see  how  other  factors  affect 


15  Paul  W.  Riley,  David  J.  Dalby,  and  Elizabeth  A.  Turner,  “Making  Acute  Hospital  Exercises  More 
Realistic  without  Impacting  on  Healthcare  Delivery,”  Journal  of  Business  Continuity  &  Emergency 
Planning  6,  no.  2  (2013):  143-150. 

15  Philip  van  Notten,  “Scenario  Development:  A  Typology  of  Approaches,”  in  Think  Scenarios, 
Rethink  Education  (Paris:  OECD  Publishing,  2006), 

http://www.oecd.org/site/schoohngfortomorrowknowledgebase/futuresthinking/scenarios/37246431.pdf 
12  Kees  van  der  Heiiden,  Scenarios:  The  Art  of  Strategic  Conversation  (Hoboken,  NJ:  John  Wiley  & 
Sons,  2005),  113. 

1^  Philip  van  Notten,  “Writing  on  the  Wall:  Scenario  Development  in  Times  of  Discontinuity”  (Boca 
Raton,  FL:  Dissertation.com,  2004). 

m  Van  Notten,  “Scenario  Development.” 

20  Van  der  Heijden,  Scenarios,  116. 


7 


expectations. 21  How  individual  agency  capabilities  are  stressed  and  how  a  variety  of 
“operational  circumstances  would  stress  those  capabilities  in  very  different  ways”22 

B,  INTELLIGENCE  SHARING 

An  understanding  of  what  constitutes  intelligence  is  important  before  examining 
intelligence  sharing.  In  Wanted:  A  Definition  of  Intelligence,  Michael  Warner  asks, 
“Without  a  clear  idea  of  what  intelligence  is,  how  can  we  develop  a  theory  to  explain 
how  it  works?”23  Warner’s  article  includes  definitions  of  intelligence  dating  back  to  the 
1950s.  Most  include  words  like  clandestine,  counterintelligence,  foreign  and  secret.  The 
FBI  takes  a  broader,  less  cloak-and-dagger  approach  to  an  intelligence  definition  and 
describes  it  this  way:  “Intelligence  is  information  that  has  been  analyzed  and  refined  so 
that  it  is  useful  to  policymakers  in  making  decisions — specifically,  decisions  about 
potential  threats.  ”2^  These  threats  can  be  in  any  area  of  homeland  security  including  the 
local  level.  Expanding  upon  that  definition,  the  FBI  website  includes  the  following: 

•  Intelligence  is  a  product  that  consists  of  information  that  has  been  refined 
to  meet  the  needs  of  policymakers. 

•  Intelligence  is  also  a  process  through  which  that  information  is  identified, 
collected,  and  analyzed. 

•  And  intelligence  refers  to  both  the  individual  organizations  that  shape  raw 
data  into  a  finished  intelligence  product  for  the  benefit  of  decision  makers 
and  the  larger  community  of  these  organizations23 

Articles  have  addressed  the  desire  and  need  to  share  intelligence.  In  a  paper  for 
the  George  Washington  Law  Review,  Nathan  Alexander  Sales,  assistant  professor  of  law 
at  the  George  Mason  University  School  of  Law,  writes  about  intelligence  sharing  among 


21  Tony  Kippenberger,  “Pitfalls  in  Scenario  Planning,”  The  Antidote  4,  no.  4  (1999):  32-33. 

22  Paul  K.  Davis,  “Uncertainty-Sensitive  Planning,”  in  New  Challenges,  New  Tools  for  Defense 
Decision  making,  ed.  Stuart  Johnson,  Martin  Libicki,  and  Gregory  F.  Treverton  (Washington,  DC:  RAND, 
2003),  131-156, 

http://www.prgs.edu/content/dam/rand/pubs/monograph_reports/MR1576/MR1576.ch5.pdf 

23  “Wanted:  A  Definition  of  ‘Intelligence’  “  Central  Intelligence  Agency,”  accessed  February  4,  2014, 
https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi- 

studi  e  s/studie  s/vo  14  6no  3  /artic  le02 .  html. 

24  “Intelligence  Defined,”  FBI,  accessed  February  9,  2014,  https://www.fbi.gov/about- 
us/intelligence/defined. 

25  Ibid. 
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the  intelligence  community.  He  writes  that  intelligence  guidelines  in  place  often  end  up 
erecting  a  “wall”  that  discourages  sharing.  26  He  suggests  that  the  wall  “was  not  intended 
to  be  impregnable”  and  that  there  exists  a  quid  pro  quo  whereby  agencies  could  trade 
information  on  a  mutually  beneficial  basis.  22 

In  United  States  and  Israeli  Homeland  Security:  A  Comparative  Analysis  of 
Emergency  Preparedness  Efforts,  Consuela  Pockett  writes  about  the  cooperation  of 
intelligence  resources  within  the  Israeli  security  mission.  If  we  examine  intelligence 
sharing  in  other  countries,  the  Israeli  model  stands  out  as  one  that  provides  citizens 
security  through  multi-layer  cooperatives,  where  the  military,  security,  and  intelligence 
forces  work  together  to  supply  an  outer  layer  of  protection  through  proactive  operations 
against  terrorist  activities. 28 

A  2006  Homeland  Security  Affairs  Journal  article  looks  at  the  negative  effects  of 
not  sharing  intelligence.  Authors  Amy  K.  Donohue  and  Robert  V.  Tuohy  ask  the  following 
questions  regarding  lessons  not  learned  after  a  disaster. 

1 .  Is  it  true  that  lessons  recur? 

2.  What  lessons  are  persistently  identified? 

3.  Why  do  these  lessons  continue  to  be  identified  as  important? 

4.  Why  are  these  lessons  so  hard  to  learn?  (That  is,  why  do  agencies  have 
difficulty  devising  and  implementing  corrective  actions  once  lessons  are 
identified?) 

5.  How  do  lessons-learned  processes  work? 

6.  How  can  they  be  improved?29 


26  Nathan  Sales,  “Share  and  Share  Alike:  Intelligence  Agencies  and  Information  Sharing,”  George 
Mason  Law  &  Economics  Research  Paper,  no.  09-24  (2009):  287. 

22  Sales,  “Share  and  Share  Alike,”  288. 

28  Consuela  B.  Pockett,  United  States  and  Israeli  Homeland  Security:  A  Comparative  Analysis  of 
Emergency  Preparedness  Efforts.  (Maxwell  Air  Force  Base,  AL:  Counterproliferation  Center,  August 
2005). 

29  Amy  K.  Donahue  and  Robert  V.  Tuohy,  “Lessons  We  Don’t  Learn:  A  Study  of  the  Lessons  of 
Disasters,  Why  We  Repeat  Them,  and  How  We  Can  Learn  Them,”  Homeland  Security  Affairs  2,  no.  2 
(July  2006),  http://calhoun.nps.edu/public/handle/10945/25094. 
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Donahue  and  Tuohy  suggest  that  the  intelligenee  gathered  after  a  natural, 
aeeidental  or  man-made  disaster  is  not  being  shared  beeause  “lessons  are  too  often 
isolated  and  perishable,  rather  than  generalized  and  institutionalizedf’^o 

C.  SOCIAL  ISSUES  AFFECTING  DISASTER  RESPONSE 

Enrieo  Quarantelli,  Professor  Emeritus  and  founding  direetor  of  Disaster  Researeh 
Center  at  the  University  of  Delaware,  is  widely  known  as  one  of  the  founding  seholars  of 
the  soeial  science  of  disasters.  His  involvement  in  the  area  dates  back  to  1949  when  he 
participated  in  the  first  systematic  disaster  field  studies  as  a  researcher  in  the  National 
Opinion  Research  Center  (NORC)  team  at  University  of  Chicago.  3 1  Quarantelli  studies 
the  similarities  in  planning  for  natural  and  manmade  disasters  and  notes  that  one  method 
of  defining  a  disaster  is  based  on  the  notion  that  an  event  is  only  a  “disaster”  if  the 
demands  created  by  it  exceed  the  community’s  capacity  for  dealing  with  it.  Quarantelli 
calls  this  an  “imbalance  in  the  demand-capability  ratio  in  a  crisis  situation.”32  Quarantelli 
implies  that  it  is  our  level  of  preparedness  that  dictates  whether  or  not  a  scenario  becomes 
a  disaster  more  than  the  mechanism  of  disaster. 

According  to  the  9/11  Commission  report,  in  the  aftermath  of  the  September  11 
attacks,  the  EDNY  was  almost  forced  to  accept  the  use  of  ICS,  the  scale  of  the  response 
area  was  simply  too  large  to  manage  any  other  way.  33 

The  preparedness  of  the  nation,  post-9/1 1,  was  strongly  focused  on  terrorism.  In  a 
post-Katrina  world,  the  shift  was  toward  planning  for  natural  disasters.  34  The  result  is  a 
realization  that  the  source  of  a  disaster  is  not  nearly  as  important  as  a  community’s 
capacity  to  prepare  and  respond. 


30  Ibid. 

31  “Faculty  and  Staff  Bios,”  University  of  Delaware,  DRC-Disaster  Research  Center,  accessed  August 
29,  2013,  http://www.udel.edu/DRC/aboutus/bios/Quarantelli.html. 

32  Enrico  L.  Quarantelli,  What  Is  a  Disaster?  An  Agent  Specific  or  an  All  Disaster  Spectrum  Approach 
to  Socio-behavioral  Aspects  of  Earthquakes?  (Newark,  DE:  Disaster  Research  Center,  1981),  453-78. 

33  National  Commission,  The  9/1 1  Commission  Report. 

34  White  House,  The  Federal  Response  to  Hurricane  Katrina:  Lessons  Learned  (Washington, 
DCiWhite  House,  2006),  12. 
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Historically,  emergency  management  has  been  viewed  as  the  responsibility  of  law 
enforcement  and  fire  departments,  with  support  in  the  event  of  a  major  catastrophe  from 
public  health  and  civil  defense  organizations. ^ 5  Collaboration  for  homeland  seeurity  can 
be  defined  as  “ageneies,  organizations,  and  individuals  from  many  tiers  of  public  and 
private  seetors,  working,  training,  and  exereising  together  for  the  eommon  purpose  of 
preventing  terrorist  threats  to  people  and  property.”36  Publie  and  private  partnerships,  as 
seen  in  the  2001  artiele  in  Disaster  Prevention  and  Management  titled  “Disasters  and 
eommunities:  vulnerability,  resilience  and  preparedness,”^^  may  be  a  good  indieator  of 
the  level  of  co-operation  and  assistance  available  within  a  community.  This  can  provide  a 
measure  of  the  likelihood  of  the  success  of  mitigation  strategies  that  require  colleetive 
and  eoordinated  action  being  adopted  and  implemented.  Many  of  the  papers  reviewed 
here  laud  the  benefits  of  eooperative  efforts  though  few  of  these  studies  have  emphasized 
the  importanee  of  the  application  of  strategic  management  to  emergency  management.  39 

Several  articles  address  soeial  issues  as  affeeting  disaster  response  and 
management.  Robert  Kates,  who  helped  establish  the  International  Initiative  for  Science 
and  Sustainability,  notes  that  “the  way  men  view  the  risks  and  opportunities  of  their 
uncertain  environments  plays  a  signifieant  role  in  their  decisions  as  to  resource 
management.”40  Essentially  stating  that  an  inaccurate  assessment  of  risk  will  result  in  an 
inappropriate  level  of  preparedness,  Kates  implies  that  the  community  needs  awareness 
of  risk  before  preparedness  measures  can  begin.  This  opens  the  study  of  sociologieal 
differenees  between  agencies  and  the  willingness  to  recognize  the  benefit  of  training  eo- 
operatively.^^i  David  Alexander,  professor  and  ehief  senior  seientist  at  the  Global  Risk 


33  Sang  Ok  Choi,  “Emergency  Management:  Implications  from  a  Strategic  Management  Perspective,” 
Journal  of  Homeland  Security  and  Emergency  Management  5,  no.  1  (2008):  2. 

3®  William  V.  Pelfrey,  “The  Cycle  of  Preparedness:  Establishing  a  Framework  to  Prepare  for  Terrorist 
Threats,”  Journal  of  Homeland  Security  and  Emergency  Management  2,  no.  1  (2005):  7. 

3^  Douglas  Paton  and  David  Johnston,  “Disasters  and  Communities:  Vulnerability,  Resilience  and 
Preparedness,”  Disaster  Prevention  and  Management  10,  no.  4  (2001):  270-277. 

38  Ibid.,  274. 

39  Choi,  “Emergency  Managem,”!. 

R.W.  Kates,  “Hazard  and  Choice  Perception  in  Flood  Plain  Management,”  Research  Paper  No.  78, 
(Chicago:  University  of  Chicago  Press,  1962). 

Geoff  O’Brien  et  at,  “Approaching  Disaster  Management  through  Social  Learning,  ”  Disaster 
Prevention  and  Management  19,  no.  4  (2010):  498-508. 
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Forum  in  Davos  Switzerland, "^2  identifies  six  approaehes  to  disaster  researeh 
(geographieal,  anthropologieal,  soeiological,  developmental,  medieal  and  teehnieal) 
though  he  considers  geography  and  sociology  to  be  the  most  important.43  He  asserts  that 
the  geographical  approach  focuses  on  human-environment  interactions,  whereas  the 
sociological  approach  has  as  its  premise  that  “disasters  are  social  events  that  reflect  the 
ways,  we  live  and  structure  our  societies  and  communities. ”44  Simply  stated,  this  idea 
leads  into  the  study  of  the  predictive  nature  of  humans  based  on  physical  surroundings 
versus  relationships. 

Literature  addressing  the  sociological  needs  for  and  reluctance  to  exposing  one’s 
agency  to  vulnerabilities45  emphasizes  the  advantages  of  local  jurisdictions  conducting 
Threat,  Risk,  and  Vulnerability  Assessments  (TRVAs)  to  aid  in  the  effective  allocation  of 
resources. 46  Kerstin  Eriksson  identifies  a  reluctance  to  accept  knowledge  transfer47  if  the 
source  is  perceived  as  having  originated  outside  a  particular  group  or  profession.  In  short, 
one  group  may  be  reluctant  to  openly  share  or  receive  knowledge  with  another  simply 
because  of  organizational  biases.  48 

In  the  area  of  knowledge  transfer,  Eriksson  suggests  that  knowledge  from 
previous  experience,  applied  to  a  current  situation,  was  of  major  significance  in 
developing  emergency  response  capacity49  when  researching  her  case  study.  She  asks  if  a 


42  David  E.  Alexander,  “Disaster  Planning  and  Emergency  Management”  Disaster  Planning  and 
Emergency  Management  Blog,  February  10,  2011,  http://emergency- 
planning.blogspot.eom/2011/02/offprints-available.html. 

43  David  Alexander,  Natural  Disasters  (London:  UCL  Press,  1993),  13. 

44  Ibid.,  554. 

43  Allan  Turner,  Kenneth  Glantz,  and  Gall,  Julie  Gall,  “A  Practitioner-Researcher  Partnership  to 
Develop  and  Deliver  Operational  Value  of  Threat,  Risk  and  Vulnerability  Assessment  Training  to  Meet  the 
Requirements  of  Emergency  Responders,”  Journal  of  Homeland  Security  and  Emergency  Management  9, 
no.  2,  article  10. 

46  John  Mueller.  “Assessing  Measures  Designed  to  Protect  the  Homeland.”  Policy  Studies  Journal  38, 
no.  1  (2010):  1-21. 

42  Kerstin  Eriksson,  “Knowledge  Transfer  between  Preparedness  and  Emergency  Response:  A  Case 
Study P  Disaster  Prevention  and  Management  18,  no.  2  (2009):  162-169. 

48  Joseph  W.  Pfeifer,  “Command  Resiliency:  An  Adaptive  Response  Strategy  for  Complex  Incidents” 
(Master’s  thesis.  Naval  Postgraduate  School,  2005). 

49  Eriksson,  “Knowledge  Transfer,”  167. 
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well-designed  analytie  preparedness  plan  applied  to  effeetive  knowledge  transfer  would 
have  similar  benefits. 

In  an  artiele  about  railway  emergeney  preparedness,  Gesine  Hofinger,  Robert 
Zinke  and  Stefan  Strohsehneider,  psyehologists  from  the  Department  of  Intereultural 
Business  Communication,  Friedrich-Schiller-University  in  Jena,  Germany,  explain  the 
psychological  demands  associated  with  disaster  management.  The  trio  studied  the 
behavior  of  employees  and  civilians  for  the  duration  of  a  planned  evacuation  during  a 
simulated  below  grade  train  accident.  The  impact  of  a  quickly  unfolding,  dynamic  event 
and  the  presence  of  the  “fight  or  flight”  tendency  hinders  the  processes  of  conscious 
thinking  and  problem  solving.  Their  analysis — ’’weighing  different  options  for  action, 
asking  critical  questions — is  nearly  impossible  while  a  strong  tendency  toward  ad  hocism 
can  be  observed,”  describes  how  emotion  can  lead  to  deviation  from  plans  and  should 
be  taken  into  consideration  during  the  formation  of  these  plans. 

Thomas  Drabek,  in  The  Human  Side  of  Disaster,  explains  the  predictive  nature  of 
crowds  and  individuals  during  an  evacuation.  Drabek  has  written  or  co-written  more  than 
100  book  chapters  and  journal  articles  and  27  books  including  The  Human  Side  of 
Disaster  (2009)  and  Human  System  Responses  to  Disaster:  An  Inventory  of  Sociological 
Findings  (1986).  He  served  as  the  co-editor  of  the  International  Journal  of  Mass 
Emergencies  and  Disasters  (1986-1990)^1  Drabek  points  out  that  our  society  is  more 
susceptible  to  disaster  because  of  our  developing  technologies.  ^2  items  that  create 

ease  in  our  lives  simultaneously  create  increased  threats. 

D,  ORGANIZATIONAL  LEARNING 

Much  has  been  written  on  the  learning  patterns  of  organizations,  both  positive  and 
negative.  In  their  two-part  article  titled  Aptitude  for  Destruction,  Brian  Jackson  et  ah, 
discuss  the  learning  and  improvement  plans  of  terrorist  organizations  and  look  at  the 

Gesine  Hofinger,  Robert  Zinke,  and  Stefan  Strohsehneider,  “Role-specific  Psychological 
Requirements  in  Preparing  Public  Transportation  Staff  for  Disaster  Response,”  Disaster  Prevention  and 
Management  20,  no.  4  (2011):  398-412. 

^1  “Faculty  and  Staff  Bios,  University  of  Delaware,  DRC -Disaster  Research  Center,”  accessed  August 
29,  2013,  http://www.udel.edu/DRC/Alumni%20Bio%27s/Alumni%20Bio%20Pages/Drabek.html. 

52  Thomas  E.  Drabek,  The  Human  Side  of  Disaster  (Boca  Raton,  FL:  CRC  Press,  2013)  31. 
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variety  of  improvement  strategies  for  different  types  of  terrorist  groups.  A  pattern  of 
adapting  and  overcoming  is  necessary  for  groups  to  learn  both  from  successes  and 
failures.  jt  is  offered  in  the  article  how  anti-terrorist  movements  can  be  coordinated  by 
understanding  the  ability  to  improve  of  these  terrorist  groups.  The  second  part  of  the 
document  looks  specifically  at  five  groups  and  their  unique  styles  of  organizational 
learning.  It  provides  an  in-depth  explanation  each  group’s  ideology,  and  how  each  has 
improved  tactically  and  strategically  or  failed  to  learn  from  opportunities.^'^ 

Another  way  to  look  at  how  groups  behave  and  learn  can  be  found  in  the  ‘writings 
of  Martin  Fishbein  and  leek  Ajzenand  and  the  theory  of  reasoned  action.  theory 

states  that  “behavior  is  rational  and  is  dependent  on  the  individual’s  analysis  of  available 
information.”^^  However,  in  their  article  “A  Conceptual  Model  of  Gambling  Behavior: 
Fishbein’s  Theory  of  Reasoned  Action,”  Wm.  Theodore  Cummings  and  William  Corney 
apply  the  theory  to  explain  gambling  patterns  and  behaviors;  the  research  is  applicable  to 
organizational  expectations.  That  is  to  say  that  an  organizations  learning  is  dependent  on 
the  information  available  and  obtained.  This  would  be  true  in  a  simple  or  chaotic 
situation.  These  situations  will  likely  cycle  between  complicated,  complex  and  chaotic, 
with  the  restoration  of  simplistic  as  a  goal.  Decision  makers  will  be  expected  to  re¬ 
evaluate  the  situation  throughout  relying  on  the  expertise  and  experience  of  personnel. 
Groups  will  analyze  the  information  available  to  them  and  select  the  behavior  that  is 
optimal. Groups  may  also  make  decisions  based  on  incorrect  information  or  a 
misinterpretation  of  it.  Poker  professional  Annie  Duke  commented  on  decision  making 


Brian  A  Jackson  et  at,  Aptitude  for  Destruction:  Organizational  Learning  in  Terrorist  Groups  and 
Its  Implications  for  Combating  Terrorism  Volume  1  (Santa  Monica,  CA:  RAND,  2005), 
http://www.rand.org/publications/MG/MG331/. 

Brian  A  Jackson  et  al..  Aptitude  for  Destruction:  Case  Studies  of  Organizational  Learning  in  Five 
Terrorist  Groups,  Volume  2  (Santa  Monica,  CA:  RAND,  2005), 
http://www.rand.org/publications/MG/MG331/. 

Martin  Fishbein  and  Icek  Ajzen,  Predicting  and  Changing  Behavior:  The  Reasoned  Action 
Approach  (New  York:  Psychology  Press,  2010). 

Wm.  Theodore  Cummings  and  William  Comey,  “A  Conceptual  Model  of  Gambling  Behavior: 
Fishbein’s  Theory  of  Reasoned  Action,”  The  Journal  of  Gambling  Behavior  3,  no.  3  (Fall  1987):  190-201. 

David  J.  Snowden  and  Mary  E.  Boone,  “A  Leader’s  Framework  for  Decision  Making.”  Harvard 
Business  Review  85,  no.  1 1  (November  2007):  68. 

Monte  M.  Page  and  Carolyn  Wood  Sherif,  eds.  Nebraska  Symposium  on  Motivation 1979:  Beliefs, 
Attitudes,  and  Values  (Lincoln,  NE:  University  of  Nebraska  Press,  1980)  190. 
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saying,  “Each  decision  should  be  made  based  only  on  the  math  at  the  deeision-making 
point.”  While  it  ean  be  diffieult  to  walk  away  from  a  project  after  eommitting  extensive 
resourees,  often  it  is  the  wisest  move.^9 

As  organizations  learn  from  outside  influenees,  they  should  likewise  be  able  to 
learn  from  eaeh  other.  Jeanne  B.  Stineheomb  and  Franeisco  Ordaz  discuss  the  integration 
of  fire  and  law  enforeement  ageneies  in  an  effort  to  streamline  operations  for  the 
municipality,  understanding  that  sueeessful  eorporations  view  organizational  eulture  is 
as  the  “major  distinguishing  feature  ...  their  most  important  eompetitive  advantage,  [and] 
the  most  powerful  faetor  they  all  highlight  as  a  key  ingredient  in  their  sueeess.’’^^ 

It  is  important  for  organizations  to  benefit  not  only  from  sueeessful  endeavors,  but 
to  take  advantage  of  the  lessons  learned  when  things  do  not  go  well.  After  a  2006  West 
Virginian  mining  disaster,  Wyoming  Senator  Mike  Enzi  made  the  following  statement: 
“We  must  draw  upon  the  experienee  of  these  reeent  tragedies  to  find  better  ways  to 
respond  to  mining  aeeidents  so  that  no  other  family  is  left  without  a  father,  a  brother,  or  a 
son.”62  A  2009  artiele  in  Organization  Science  addresses  organizational  learning  by 
asking,  “Do  organizations  learn  vieariously  from  disasters  experienced  by  others?”®^ 
Suggesting  that  perhaps  if  a  tragedy  befalls  one  ageney,  only  that  ageney  will  benefit 
from  the  lessons  learned,  where,  ideally,  peripheral  ageneies  should  take  advantage  of  the 
tragedy  to  prevent  similar  disasters  within  their  own  arena.  The  artiele  goes  on  to  look  at 
how  organizations  tend  to  “improve  their  performanee  in  a  domain  as  they  gain 
experienee  in  that  domain.  ”^4 


Annie  Duke,  “Sunk  Cost,”  Bluff  Magazine,  April  2007,  http://www.bluff.com/magazine/sunk-cost- 

8962/. 

Jeanne  B.  Stineheomb  and  Francisco  Ordaz,  “The  Integration  of  Two  ‘Brotherhoods’  into  One 
Organizational  Culture:  A  Psycho-Social  Perspective  on  Merging  Police  and  Fire  Services,”  Public 
Organization  Review  7,  no.  2  (January  30,  2007). 

Kim  S.  Cameron  and  Robert  E.  Quinn,  Diagnosing  and  Changing  Organizational  Culture :  Based 
on  the  Competing  Values  Framework,  rev.  ed  (San  Francisco:  Jossey-Bass,  2006),  5. 

The  State  of  Mine  Safety  and  Flealth:  Flearing  of  the  Committee  on  Health,  Education,  Labor,  and 
Pensions,  United  States  Senate,  109th  Cong.,  2nd  Session  (2006). 

Peter  M.  Madsen,  “These  Lives  Will  Not  Be  Lost  in  Vain:  Organizational  Learning  from  Disaster  in 
U.S.  Coal  Mining,”  Organization  Science  20,  no.  5  (October  2009):  861-875,  doi:10.1287/orsc. 1080. 0396. 
64  Ibid. 
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Peter  M.  Senge  attributes  this  to  a  “Mental  Models”  all  of  us  earry  within 
ourselves.  Senge  is  eredited  with  the  ieeberg  model,  whieh  breaks  “knowledge  into 
three  eategories:  events,  pattern  and  structure. This  is  the  basis  for  using  scenarios  as 
part  of  training.  Not  to  predict  actual  events  but  to  better  understand  the  basis  for 
expected  actions.  Scenario  planners  begin  with  the  idea  that  there  is  much  more  to 
analysis  than  simply  reporting  and  mimicking  events  as  they  occurred. 

Diane  Vaughn  of  Boston  College’s  Department  of  Sociology,  explores  the 
negative  side  of  organizations  in  The  Dark  Side  of  Organizations:  Mistake,  Misconduct, 
and  Disaster.  Vaughn  suggests  that  organizations  may  perform  detrimentally  to  positive 
expectations  by  conforming  to  organizational  behaviors.  This  may  seem  to  run  contrary 
to  what  Fishbein  says  regarding  expected  action,  but  may  be  just  as  predictable.  Vaughn 
uses  the  term  organizational  deviance  and  defines  it  as  “an  event,  activity,  or 
circumstance,  occurring  in  and/or  produced  by  a  formal  organization  that  deviates  from 
both  formal  design  goals  and  normative  standards  or  expectations,  either  in  the  fact  of  its 
occurrence  or  in  its  consequences,  and  produces  suboptimal  outcome. 

The  psychological,  organizational,  social  and  cultural  barriers  to  change  can  have 
a  very  powerful  effect  on  whether  a  change  model  will  be  accepted  or  rejected  and 
applied  by  the  larger  whole.  Tension  develops  any  time  new  strategies  or  ideas  that 
conflict  with  established  culture.  A  group  may  find  difficulties  expanding  its  accepted 
sphere  of  influence  because  of  group  solidarity  or  its  rejection  of  outsiders.  This  is 
consistent  in  what  was  presented  by  Henri  Tajfel  in  his  Social  Identity  writings. The 
organization  may  view  outsiders  as  a  threat  and  seek  to  protect  its  position.  It  may 
struggle  with  differing  opinions  of  needs  internally  as  well.  Because  of  “Structural 


Peter  M.  Senge,  “Mental  Models,”  Strategy  &  Leadership  20,  no.  2  (1992):  4-44. 

Peter  M.  Senge,  The  Fifth  Discipline  (New  York:  Random  House  Audio,  1994). 

Van  der  Heijden,  Scenarios,  104. 

Diane  Vaughan,  “The  Dark  Side  of  Organizations:  Mistake,  Misconduct,  and  Disaster,”  Annual 
Review  of  Sociology,  1999,  271-305. 

69  Ibid. 

70  Ibid. 

71  Henri  Tajfel,  Social  Identity  and  Intergroup  Relations  (Cambridge:  Cambridge  University  Press, 
2010). 
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secrecy”  some  at  differing  hierarchical  levels  within  the  organization  may  advocate  for 
change  while  others  resist.  ^2  Tension  can  be  created  when  changes  threaten  to  infiltrate 
an  organization’s  way  of  doing  things,  strategically,  structurally  or  culturally.  ^3 

The  four  psychological  barriers  presented,  “perception,  homeostasis,  conformity 
and  commitment,  and  personality  factors”  ^4  are  offered  as  issues  that  can  affect  an 
organization’s  ability  not  only  to  accept  potential  new  ideas  and  advance  but  its  capacity 
to  develop  competitively  with  other  organizations;  a  shortcoming  that  could  result  in  the 
termination  of  an  organization.  Innovation  is  often  a  barrier  in  and  of  itself  though  there 
is  often  an  advantage  when  innovation  is  developed  from  within  an  established  agency.  ^3 
Creating  a  need  for  new  technology,  or  in  this  case,  a  new  system,  is  often  met  with 
skepticism.  Organizations  may  be  reluctant  to  accept  a  new  system  on  top  of  a  system 
they  already  understand  and  believe  in.  However,  according  to  The  Innovator 's  Dilemma, 
if  we  can  create  a  separate  organization  and  embed  it  among  potential  new  customers  that 
need  the  new  process,  we  can  anticipate  success  as  we  move  the  innovation  forward.  ^6 

E,  COLLABORATION 

At  the  municipal  level,  collaboration  in  the  areas  of  preparedness  and  disaster 
management  is  not  only  a  practical  use  of  resources  but  also  a  far  more  efficient 
application  of  them.  Any  municipality  needs  a  strategy  in  order  to  prepare  itself  for 

emergencies.  22 


22  Diane  Vaughan,  The  Challenger  Launch  Decision  Risky  Technology,  Culture,  and  Deviance  at 
NASA  (Chicago:  University  of  Chicago  Press,  1996). 

23  “The  Resistance  to  Change,”  accessed  February  4,  2014, 
https://sites.google.com/site/theresistancetochange/home. 

24  Ibid. 

23  Christensen,  The  Innovator’s  Dilemma,  84-88. 

26  Ibid.,  90. 

22  Steven  Weiss,  “Review  of  Communicating  Emergency  Preparedness:  Strategies  for  Creating  a 
Disaster  Resilient  Public,”  Journal  of  Homeland  Security  and  Emergency  Management  6,  no  1,  article  70. 
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Many  documents  address  the  failures  associated  with  attacks  of  September  11, 
2001,  as  well  as  the  response  to  Hurricane  Katrina  in  2005.^*  The  lessons  learned  have 
steered  cooperative  training  toward  a  “whole-community”  approach  to  preparedness. 
By  learning  from  these  events  and  involving  multiple  agencies  in  planning,  we  can  ensure 
that  redundancies  best  serve  the  greater  good  and  help  guide  decision  makers. 

There  are  a  number  of  barricades  related  to  inter-organizational  collaboration. 
Some  research  indicates  impracticality  to  combined  training  based  on  safety  factors,  cost 
and  productivity. These  are  often  case  by  case  and  are  rarely  systemic.  In  Building 
Collaborative  Capacity:  An  Innovative  Strategy  for  Homeland  Security  Preparedness 
the  authors  point  to  five  organizational  design  components  based  on  Jay  Galbraith’s  Star 


^^David  M.  Simpson,  “Disaster  Preparedness  Measures:  A  Test  Case  Development  and  Application,” 
Disaster  Prevention  and  Management  17,  no.  5  (2008):  645-661. 

Donahue  et  at,  “The  All  Needs  Approach  to  Emergency  Response.” 

80  Peter  J.  Smith,  Ian  Robertson,  and  Lyn  Wakefield,  “Developing  Preparedness  for  Flexible  Delivery 
of  Training  in  Enterprises,”  Journal  of  Workplace  Learning  14,  no.  6  (2002):  222-232. 

81  Susan  Page  Hocevar,  Gail  Farm  Thomas,  and  Erik  Jansen.  “Building  Collaborative  Capacity:  An 
Innovative  Strategy  for  Homeland  Security  Preparedness.”  in  Advances  in  Interdisciplinary  Studies  of  Work 
Teams  (Bingley,  UK:  Emerald,  2006),  255-274. 
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framework  for  organization  design.  *2  These  are:  purpose  and  strategy,  strueture, 
proeesses,  ineentives  and  people.  These  are  then  divided  by  sueeess  faetors  and  barrier 
faetors.  For  example,  among  ineentives,  leadership  support  and  eommitment  is  listed  as  a 
sueeess  faetor  while  eompetition  for  resourees  is  seen  as  a  barrier  faetor.  The  star  model 
depiets  a  support  system  that  is  key  in  eollaboration;  a  support  system  that  needs  to  exist, 
well  before  the  idea  of  interageney  preparedness  efforts  is  explored.  Dealing  with  hurdles 
in  advanee  of  their  effeet  on  planning  will  minimize  their  impaet  during  the  onset  of  the 
disaster.  Kineaid  suggests  that  there  is  a  need  for  the  training  of  personnel  from  different 
ageneies,  with  different  professions  and  standard  operating  proeedures,  to  understand 
eaeh  other’s  methods  and  proeedures  in  order  to  eoordinate  response  most  eflioiently.83 
Some  of  this  inter- ageney  training  ean  be  eondueted  using  web-based  teehniques,^^ 
whieh  exposes  eaeh  ageney  to  the  eapabilities  of  the  other,  fostering  a  network  of  sharing 
as  well  as  limiting  the  need  for  eoordinating  several  ageneies’  sehedules. 

F.  CONTENT  AND  THEMATIC  ANALYSIS 

In  order  to  develop  a  data  series  for  this  thesis,  it  beeame  neeessary  to  gain  an 
understanding  of  a  method  to  analyze  data.  Researeh  in  the  areas  of  Content  Analysis  and 
Thematie  Analysis  shed  some  light  on  an  important  researeh  teehnique.  In  Klaus 
Krippendorf  s  Content  Analysis:  An  Introduction  to  its  Methodology  the  author  explains 
the  method  used  to  extraet  numerie  value  from  text.  Using  an  example  from  World  War 
II  where  Allied  analysts  were  able  to  extrapolate  an  aeeurate  launeh  date  for  Germany’s 
V-weapon  based  on  the  monitoring  and  analysis  of  Joseph  Goebbel’s  speeehes.^^  The 
example  illustrates  the  effeetiveness  of  the  method  prior  to  the  widespread  availability  of 
eomputers  for  analytieal  purposes. 


^2  Galbraith,  Jay  R.  Designing  Organizations:  An  Executive  Briefing  On  Strategy,  Structure,  and 
Process  (San  Francisco:  Jossey-Bass),  1995. 

Peter  Kincaid,  Joseph  Donovan  and  Beth  Pettitt,  “Simulation  Techniques  for  Training  Emergency 
Response,”  International  Journal  of  Emergency  Management  1,  no.  3  (2003). 

Jenvald,  J.,  Morin,  M.  and  J.  Peter  Kincaid,  “A  Framework  for  Web-based  Dissemination  of  Models 
and  Lessons  Learned  from  Emergency-response  Exercises  and  Operations,”  International  Journal  of 
Emergency  Management  1,  no.  1  (2001):  82-94. 

Klaus  Krippendorff,  Content  Analysis:  An  Introduction  to  Its  Methodology  (Beverly  Hills:  SAGE, 
1980),  17. 
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When  analyzing  data  for  interpretive  results,  there  ean  be  “eonsiderable  overlap 
among  available  qualitative  approaehes  in  terms  of  methods,  proeedures,  and 
teehniques.”*^  This  is  true  in  the  area  of  partieipant  observation  whieh  was  a  signifioant 
input  to  the  methodology.  As  AARs  are  developed  with  eontribution  from  partieipants,  an 
added  perspeetive  is  attaehed  to  the  data.  This,  eoupled  with  the  eontributions  from 
“objeetive  observers”  leaving  the  analyst  with  additional  interpretive  ehallenges.^^ 

As  text  is  interpreted,  its  elassifieation,  now  by  eomputer,  leads  to  a  more  aeeurate 
translation  into  a  data  set.  Robert  Philip  Weber  writes 'm  Basic  Content  Analysis  about  the 
elassifications  of  data,  where  the  use  of  several  elassifications  and  sub-order 
elassifieations  may  lead  to  diffieulties  based  on  multiple  meanings  of  the  same  text.^^  In 
ehoosing  a  methodology  for  this  thesis  that  would  allow  understanding  of  the  data 
presented,  qualitative  deseriptive  approaehes  sueh  as  eontent  analysis  and  thematie 
analysis  were  ehosen  beeause  they  employ  a  “relatively  low  level  of  interpretation,  in 
eontrast  to  grounded  theory  or  hermeneutie  phenomenology,  in  whieh  a  higher  level  of 
interpretive  eomplexity  is  required.” ^9 

Both  content  analysis  and  thematic  analysis  are  similar  in  that  they  analytically 
examine  narrative  materials  from  written  recommendations  by  “breaking  the  text  into 
relatively  small  units  of  content  and  submitting  them  to  descriptive  treatment,”90  which 
fit  the  data  analysis  for  this  thesis  perfectly. 

G.  LITERATURE  REVIEW  CONCLUSION 

After  a  review  of  the  literature  associated  with  exercise  design  we  are  left  to 
explain  the  connection  between  exercise  design  and  societal  biases  preventing  a 


Mojtaba  Vaismoradi,  Hannele  Turunen,  and  Terese  Bondas,  “Content  Analysis  and  Thematic 
Analysis:  Implications  for  Conducting  a  Qualitative  Descriptive  Study:  Qualitative  Descriptive  Study,” 
Nursing  &  Health  Sciences  15,  no.  3  (September  2013):  398-405,  doi:10.1 11 1/nhs.  12048. 

Beniamin  F.  Crabtree  and  William  L.  Miller,  Doing  Qualitative  Research  (Thousand  Oaks,  CA: 
SAGE,  1999),  47. 

Robert  Philip  Weber,  Basic  Content  Analysis  (Newbury  Park,  CA:  Sage  Publications,  1990),  36. 
Vaismoradi,  Turunen,  and  Bondas,  “Content  Analysis  and  Thematic  Analysis.” 

Andrew  Sparkes,  “Narrative  Analysis:  Exploring  the  Whats  and  Hows  of  Personal  Stories,”  in 
Qualitative  Research  in  Health  Care,  ed.  Immy  Holloway,  1st  ed  (Berkshire:  Open  University  Press,  2005), 
191-208. 
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realization  of  the  eollaborative  benefits.  How  one  diseipline  within  the  eommunity  may 
or  may  not  accept  exercise  design  direction  from  the  municipality.  In  this  case,  the 
FDNY,  but  under  the  proper  circumstances  any  municipal  agency  may  be  capable  of 
offering  this  assistance.  The  literature  examines  the  social  barriers  that  may  hinder 
cooperation  and  prevent  collaborative  contributions  to  after  action  reporting.  A  further 
acceptance  of  differing  thoughts  and  perspectives  needs  to  be  explored  and  some  of  these 
barriers  identified  and  minimized. 

The  literature  looked  at  the  different,  predictable  ways  groups  can  be  expected  to 
behave  during  a  disaster.  This  intelligence  should  also  be  applied  to  designing  exercises 
for  groups  clustered  for  different  reasons,  for  example,  people  grouped  geographically  in 
a  train  station  or  by  virtue  of  similar  employment  like  those  working  together  in  a  high- 
rise  office  building. 

Much  of  the  literature  reviewed  comes  from  scholarly  articles  in  emergency 
journals.  These  articles  relate  the  benefits  learned  from  the  experiences  of  the  authors. 
Articles  on  behavior,  predictability  and  organizational  biases  as  well  as  a  large  sampling 
of  AARs,  culled  for  relevance.  Further  investigation  of  the  sources  used  in  the  journal 
articles  will  apply  increased  credibility  to  the  articles  and  thoughts  expressed  therein. 
There  is  a  need  for  additional  studies  of  emergency  situations  and  examination  of  the 
results  from  earlier  literature  to  extract  relevant  knowledge  to  be  transferred  into  various 
preparedness  activities  and  future  exercises  and  response  to  crises. 

The  USFA  and  U.S.  Army’s  information  on  AARs  lays  out  a  framework  for 
comprehensive  AAR  development,  but  moving  forward,  a  greater  emphasis  needs  to  be 
directed  toward  the  inclusion  of  intelligence.  Intelligence  that  can  lead  to  the 
development  of  a  proactive  rather  than  reactive  exercise  design  process,  one  where  we 
design  exercises  based  on  what  we  should  prepare  for  rather  than  one  where  we  look 
back  on  what  we  should  have  done. 


91  Kerstin  Eriksson,  “Knowledge  Transfer  between  Preparedness  and  Emergency  Response:  A  Case 
Study  "  Disaster  Prevention  and  Management  18,  no.  2  (2009):  162-169, 


doi:10.1 108/09653560910953234. 
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III.  COLLABORATION  AND  RESISTANCE  TO  CHANGE 
IN  DISASTER  PREPAREDNESS 


It  is  a  common  defect  in  man  to  not  make  any  provision  in  the  calm 
against  the  tempest. 

— Niccolo  Machiavelli,  The  Prince 


A,  INTRODUCTION 

The  greater  Homeland  Seeurity  mission  of  eommunity  resilieney  ean  be  better 
achieved  through  the  collaborative  efforts  of  the  individual  entities  that  make  up  the 
community. 92  This  includes  stakeholders  both  public  and  private.  While  this  may  seem  a 
logical  assumption,  often  private  stakeholders  do  not  take  advantage  of  the  opportunity  to 
participate  in  training  exercises  organized  and  run  by  public  agencies.  Private 
stakeholders  often  do  not  even  exercise  with  each  other.  Municipal  agencies,  fire, 
emergency  medical,  law  enforcement,  and  public  utilities  rarely  collaborate  with  each 
other,  never  mind  with  private  entities.  Effective  collaboration  requires  both  cultural 
sensitivity  and  a  common  language. 93  “Nonetheless,  conflicts  are  inevitable,  and  some 
organizations  simply  may  be  unable  or  unwilling  to  work  with  others. ”94  The 
involvement  of  nongovernmental  participants  strengthens  the  community’s  ability  to  deal 
with  future  disasters.  The  ability  of  a  community  to  respond  to  and  recover  from  disasters 
is  not  enhanced  when  officials  preempt  or  exclude  community  involvement.  95  This 
chapter  will  examine  organizational  roadblocks  and  sociological  limitations  that  prevent 
organizations  from  attaining  a  realistic  measure  of  community  resiliency,  with  the 
ultimate  goal  being  increased  willingness  of  all  stakeholders  contributing  to  the  resiliency 
of  the  community. 


92  Hocevar,  Thomas  and  Jansen,  “Building  Collaborative  Capacity,”  255-274. 

93  William  L.  Waugh  and  Gregory  Streib,  “Collaboration  and  Leadership  for  Effective  Emergency 
Management,”  Public  Administration  Review  66,  no.  1  (2006):  131-140. 

94  Ibid. 

95  Ibid. 
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B, 


WHAT  IS  A  DISASTER? 


How  does  an  event  become  a  disaster?  ^6  Enrico  Quarantelli,  Professor  Emeritus 
and  founding  director  of  Disaster  Research  Center  at  the  University  of  Delaware  is 
widely  known  as  one  of  the  founding  scholars  of  the  social  science  of  disasters. He 
determines  whether  an  event  is  a  disaster  by  the  preparedness  level  of  the  jurisdiction  on 
the  receiving  end  of  the  “disaster.”  In  their  article  “Response  to  Social  Crisis  and 
Disaster,”  Quarantelli  and  Russell  Dynes  summarize  basic  substantive  and  structural 
trends  that  assume  that  a  “disaster  is  primarily  a  social  phenomenon  and  is  thus 
identifiable  in  social  terms. This  is  to  suggest  that  the  nature  of  the  disaster,  natural  or 
man-made,  is  not  as  significant  as  the  social  impact  when  defining  a  disaster’s  scope. 
David  Alexander,  professor  and  chief  senior  scientist  at  the  Global  Risk  Eorum  in  Davos 
Switzerland, asserts  that  the  localized  approach  focuses  on  human-environment 
interactions,  whereas  the  sociological  approach  believes  that  “disasters  are  social  events 
that  reflect  the  ways,  we  live  and  structure  our  societies  and  communities.”  Simply 
stated,  this  idea  leads  into  the  study  of  the  predictive  nature  of  humans  based  on  physical 
surroundings  versus  relationships.  As  responders,  emergency  planners  need  to  focus  less 
on  the  origin  and  more  on  the  preparedness  efforts  of  the  community  affected. 

Quarantelli ’s  argument  is  valid  that  “disasters  are  instabilities  arising  at  the 
interface  between  society  and  the  environment.” Eour  factors  must  be  considered  when 
defining  a  disaster;  “the  physical  agent,  the  physical  consequences  of  the  agent,  the  way 
in  which  the  impact  of  the  physical  agent  is  evaluated,  and  the  social  disruption  and 
social  changes  brought  about  by  the  physical  agent  and  its  impact.”  Even  where  there 
is  evidence  of  societal  improvement  driving  community-based  planning  for  disasters,  this 
has  arisen  chiefly  because  of  a  lack  of  resources  (the  term  disaster  implies  an 

Lomnitz,  “What  Is  a  Disaster,”  87-88. 

97  “Faculty  and  Staff  Bios,”  University  of  Delaware. 

98  E.L.  Quarantelli  and  Russell  R.  Dynes,  “Response  to  Social  Crisis  and  Disaster,”  Annual  Review  of 
Sociology  3  (1977):  23-49. 

99  Alexander,  “Disaster  Planning  and  Emergency  Management.” 

100  “Editorial  Board,”  Journal  of  Geography  &  Natural  Disasters,  accessed  August  8,  2013, 
http://www.omicsgroup.0rg/joumals/editorialboardJGND.php#. 

Lomnitz,  “What  Is  a  Disaster,”  88. 

102  Quarantelli  and  Dynes,  “Response  to  Social  Crisis  and  Disaster.” 

24 


103 

overwhelming  of  the  eommunity’s  eapacity  to  cope).  That  is  to  say  that  the  disaster 
exists  primarily  due  to  a  community’s  inability  to  overcome  the  challenges  presented  by 
the  event  causing  the  “disaster.”  If  a  tornado  tears  through  a  field  and  damages  no 
infrastructure,  can  it  really  be  called  a  disaster? 

In  preparing  for  or  responding  to  disaster,  the  local  government  has  the  first  line 
of  official  public  responsibility.  However,  the  affected  community  will  be  responsible  for 
itself  in  the  initial  minutes  of  any  disaster  until  help  arrives.  Longer  depending  on  the 
nature  of  impact  and  size  of  emergency  management  supports  the  local  response  effort  by 
working  with  the  public  and  private  partners  to  develop  emergency  plans  and  capabilities 
that  can  respond  to  any  natural  or  man-made  hazard  which  threatens  the  community, 
We  cannot  think  of  natural  disasters  as  being  divine  in  origin  or  intent  (“acts  of  God”)  but 
recognize  that  whether  natural  or  man-made  the  definition  of  the  disaster  is  rooted  in  the 
“relationship  between  peoples  and  their  environments.” 

C.  LESSONS  NOT  LEARNED 

Lessons  from  disasters  are  often  not  incorporated  into  wider  preparedness 
measures.  Improvements  are  often  only  accepted  as  “doing  it  better”  as  opposed  to 
“doing  it  differently. ”106  The  thought  behind  a  lessons-learned  approach  to  evaluation  is 
that  we  strive  to  improve.  We  believe  that  through  practice  we  will  continue  to  improve 
and  better  prepare  ourselves  and  those  for  whom  we  are  responsible.  By  conducting 
evaluations  and  summarizing  findings,  we  share  our  knowledge  across  a  lessons-learned 
format.  Empirical  data  should  then  show  some  form  of  consistent  improvement  or  at  least 
a  refocusing  of  efforts  toward  those  areas  where  we  are  in  the  greatest  need  of 
improvement.  However,  in  many  cases,  the  opposite  was  true.  According  to  the  data 
collected,  several  recommendations  appeared  the  most  after  real-world  events  and 


Geoff  O’Brien  et  al.,  “Approaching  Disaster  Management  through  Social  Learning,”  Disaster 
Prevention  and  Management  19,  no.  4  (2010):  498-508,  doi:10. 1108/09653561011070402. 

Hocevar,  Thomas,  and  Jansen,  “Building  Collaborative  Capacity.” 

O’Brien  et  al.,  “Approaching  Disaster  Management,”  499. 

106  Ibid. 
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exercises,  namely:  ICS  concerns,  communications,  resources,  EOC  issues,  and 
documentation. 

One  could  logically  assume  then  that  these  would  feature  prominently  as  exercise 
objectives;  however,  as  Table  1  shows,  there  was  a  tremendous  disparity  between 
recommendations  and  objectives. 


Table  1 .  Disparity  between  Recommendations  and  Objectives 


Recommendation 

Real  World 

Exercise 

Objecthe 

%  Dispa  rits^ 

ICS 

55% 

62% 

6% 

90% 

Communications 

70% 

34% 

6% 

82% 

Resources 

38% 

57% 

15% 

78% 

EOC 

48% 

62% 

15% 

39% 

Documentation 

45% 

38% 

15% 

61% 

ICS  was  included  in  recommendations  from  exercise  AARs  62%  of  the  time  but 
only  appeared  as  an  objective  in  6%  of  exercises.  It  was  a  recommendation  in  55%  of  real 
world  AARs.  This  indicates  a  gap  between  lessons  learned  and  future  planning  If 
recommendations  are  made  in  either  an  exercise  or  real  world  AAR  then  there  is  a 
responsibility  to  incorporate  it  into  training  and  eventually  include  it  as  an  objective  in 
future  exercises  in  an  attempt  to  validate  training.  By  not  addressing  it,  we  are  remaining 
complacent  and  expressing  contentment  with  the  status  quo,  despite  the  fact  that  the  very 
system  put  into  place  to  minimize  complacency  identifies  the  status  quo  as  being 
ineffective. 

D,  RESISTANCE  TO  CHANGE 

The  need  for  collaboration  often  meets  resistance  as  a  result  of  social  and 

organizational  biases.  That  is,  one  group  may  limit  their  preparedness  activities  with 

another  group  based  on  preconceived  notions.  Also,  one  group  may  exhibit  perceived 
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superiority  to  the  other  group  based  on  intergroup  relations.!*’^  Viewing  the  benefits  of 
eollaboration  as  weighted  in  favor  of  one  side  over  the  other  ereates  a  eompetitive  seareh 
for  value  equity.  FDNY  Chief  Joseph  Pfeifer  addressed  the  problems  assoeiated  with 
group  biases,  stating  that: 

Soeial  identity  that  promotes  power  of  one  organization  over  another 
organization  produees  two  soeial  outeomes  during  eomplex  ineidents. 

First,  there  is  the  ereation  of  a  positive  in-group  bias  toward  those  who  are 
part  of  the  same  group  and  a  negative  out-group  bias  against  those  who  are 
part  of  an  alternate  group.  When  providing  information  aeross  groups, 
individuals  are  prone  to  give  more  information  to  members  of  their  own 
group  and  less  to  members  outside  that  group.  Seeondly,  when  under 
stress,  individuals  feel  little  obligation  to  share  valuable  information  with 
those  outside  their  group  since  responsibility  for  acting  is  diffused  within 
their  in-group.  This  phenomenon  excludes  the  out-group  from  receiving 
information  that  may  be  vital  to  their  operation.  1^8 

This  results  in  a  siloed  approach  to  preparedness,  despite  evidence  available  to 
individual  organizations  that  collaboration  would  benefit  each  of  them.  So,  why  would 
organizations  allow  bias  to  interfere  with  safety?  We  have  a  tendency  only  to  see  the 
problems  within  our  organizations  that  we  are  looking  for.  Therefore,  additional 
perspectives  applied  to  disaster  response  and  preparedness  are  often  unwelcomed  and 
unutilized.  When  looking  at  an  organization’s  response  plan  for  various  disaster 
scenarios,  the  assumption  is  made  that  certain  sequences  of  actions/events  are  likely  to 
happen.  109  These  expectations,  and  the  assumed  secondary  and  tertiary  responses,  are 
partly  based  in  the  practices  and  routines  of  our  organizations  and  in  our  management 
strategies.  110 

Organizations  fail  at  collaboration  for  many  reasons:  organizations  have 
individual  missions  with  goals  and  incentives  that  often  conflict  with  one  another; 
agencies  often  have  histories  of  distrust  that  may  be  currently  irrelevant  but  are. 


107  Tajfel,  Social  Identity  and  Intergroup  Relations. 

10^  Joseph  W.  Pfeifer,  “Understanding  How  Organizational  Bias  Influenced  First  Responders  at  the 
World  Trade  Center”  in  Psychology  of  Terrorism  ed.  Bruce  Michael  Bongar  et  ah,  (New  York:  Oxford 
University  Press,  2007). 

109  Teemu  Reiman  and  Carl  Rollenhagen,  “Human  and  Organizational  Biases  Affecting  the 
Management  of  Safety,”  Reliability  Engineering  &  System  Safety  96,  no.  10  (October  2011):  1263-1274, 
doi:10.1016/j.ress.2011.05.010. 

1 10  Peter  Wilkin,  “The  Ideology  of  Ergonomics,”  Theoretical  Issues  in  Ergonomics  Science  1 1,  no.  3 
(2010):  230-244. 
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nonetheless,  hard  to  alter;  leaders  may  not  aetively  support  collaborative  efforts; 
and  coordination  systems  and  structures  that  might  support  collaboration  are  often 
lacking/'' 

There  can  be  expected  reluctance  to  try  new  ideas.  Those  aware  of  the 
improvements  associated  with  a  new  idea  face  the  challenge  of  “selling”  this  to  a  group 
that  may  only  be  conscious  of  the  current  practices.  The  Allegory  of  Plato’s  Cave 
illustrates  this  well;  1 12 

Plato  imagines  a  cave,  in  which  several  prisoners  are  tied  in  such  a  way 
that  they  cannot  move  or  see  anything  but  the  stone  wall  in  front  of  them. 

These  prisoners  have  been  in  this  position  their  entire  lives.  These 
prisoners  have  never  seen  outside  of  the  cave.  Behind  them  is  a  fire,  and 
between  them  and  the  fire  is  a  raised  roadway.  Other  people  walk  along 
this  walkway  carrying  shapes.  When  they  do,  the  shadows  of  the  shapes 
they  are  carrying  are  cast  on  to  the  wall.  Having  never  seen  the  real 
objects  before,  the  prisoners  believe  that  the  shadows  of  objects  were 
“real.” 

Plato  suggests  that  the  prisoners  would  begin  a  “game”  of  guessing  which 
shadow  would  appear  next.  If  one  of  the  prisoners  were  to  correctly  guess, 
the  others  would  praise  him  as  clever  and  say  that  he  were  a  master  of 
nature. 

Eventually,  one  of  the  prisoners  escapes  and  leaves  the  cave.  He  is  amazed 
at  the  world  he  discovers  outside  the  cave  and  does  not  believe  it  can  be 
real.  He  wanders  about  absorbing  the  sights  and  sounds  of  his  new 
surroundings.  As  he  becomes  accustomed  to  his  new  world,  he  realizes 
that  his  former  old  world  was  inaccurate. 

He  wonders  at  the  sun  and  starts  an  intellectual  journey  consisting  of  these 
wonderful  new  experiences.  He  realizes  also  the  game  he  had  played  with 
the  other  prisoners  was  now  valueless.  The  prisoner  returns  to  the  cave,  to 
share  his  amazing  findings  with  the  other  prisoners.  They  do  not  believe 
him  and  threaten  to  kill  him  if  he  tries  to  free  them.  1 


1 1 1  U.S.  Government  Accountability  Office,  Homeland  Security:  Management  Challenges  Facing 
Federal  Leadership  (Washington,  DCiUnited  States  Government  Accountability  Office,  December  2002). 

1 12  Plato,  The  Allegory  of  the  Cave  (Brea,  CA:  P  &  L,  2010). 

113  Ibid. 
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Figure  2.  Depiction  of  the  Cave 

Once  the  prisoner  returns  to  the  others  in  the  cave,  he  tries  to  share  his 
experiences  but  the  group  is  not  interested.  They  cannot  imagine  what  he  is  telling  them 
to  be  true  and  suggest  he  rejoin  the  group.  It  is  not  possible  for  him  to  forget  what  he  has 
seen  and  he  is  forever  influenced  by  his  experience  outside.  Organizations  react  this  way 
when  faced  with  significant  change.  Those  aware  of  the  benefits  of  the  new  proposal 
instinctively  wish  to  promote  the  idea  while  those  unfamiliar  experience  reluctance  to 
change.  “That  is  not  how  we  do  things”  is  often  the  death  knell  to  many  innovative 
suggestions. 

The  transition  towards  a  new  way  of  accomplishing  goals  takes  time  and 
should  benefit  from  innovation  and  experience.  Learning  from  events,  we 
argue,  should  be  a  catalyst  for  thinking  about  how  we  do  things  going 
forward.  Many  published  articles  are  narratives  of  a  single  disaster  written 
from  the  perspective  of  one  individual.  1 

In  the  UK,  for  example,  the  fire  and  rescue  service  has  experienced  a  successful 
modification  from  “fire  fighting”  to  “fire  prevention”  that  exhibits  the  shift  in  emphasis 
of  institutional  thinking,  from  reactive  (putting  out  fires)  to  proactive  (preventing 
fires). 

1  Erik  Auf  der  Heide,  Disaster  Response:  Principles  of  Preparation  and  Coordination  (St.  Louis, 
MO:  Mosby,  1989). 

O’Brien  et  al.,  “Approaching  Disaster  Management  through  Social  Learning.” 
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Emergency  management,  as  a  profession  has  evolved  into  a  more  collaborative 
enterprise  over  the  last  50  years.  There  has  been  a  gradual  change  away  from  the 
traditional  hierarchical  structure  toward  a  more  “dynamic  and  flexible  network  model 
that  facilitates  multi-organizational,  intergovernmental,  and  intersectoral  cooperation.” 

Professor  Arthur  Stinchcombe’s  liabilities  of  newness  theory  explains 
comparative  death  rates  of  new  and  old  organizations:  A  higher  proportion  of  new 
organizations  are  likely  to  fail  than  old.  Although  Stinchcombe  uses  liabilities  of  newness 
to  explain  the  development  of  populations  of  organizational  forms,  his  concept  also  may 
apply  to  analysis  of  cooperative  movements,  explaining  why  new  programs  or  ideas 
struggle  to  succeed,  or  even  the  problems  associated  with  transitioning  new  personnel 
into  organizations.il* 

The  Reiman  and  Rolenhagen  table  looks  at  what  they  describe  as  the  four 
thematic  areas  of  human  and  organizational  behavior.  These  are  beliefs  about  human 
behavior,  beliefs  about  organizations,  beliefs  about  information  and  safety  models, 
Reiman  and  Rolenhagen  constructed  the  information  in  Table  2  to  display  the  integration 
of  biases  and  their  negative  effect  on  the  mission  of  resilience  and  safety  demonstrating 
that  biases  should  not  be  considered  in  isolation  from  each  other — the  biases  are 
interrelated,  as  are  the  systems  they  address.  The  table  can  serve  as  a  simplified  depiction 
of  how  a  bias  in  some  area  of  safety  management  can  have  practical  consequences  for  the 
way  safety  is  managed  in  the  organization.  They  suggest  that  these  biases  can  lead  to 
decision  making  in  a  vacuum  which  breeds  complacency.  There  is  a  false  sense  of  safety 
that  develops  from  the  deficiency  of  challenges  to  our  safety;  that  a  lack  of  catastrophe 
can  be  viewed  as  a  lack  of  risk. 


116  Waugh  and  Streib,  “Collaboration  and  Leadership  for  Effective  Emergency  Management.” 

117  Ibid. 

11*  Vaughan,  “The  Dark  Side  of  Organizations,”  275. 

1 19  Reiman  and  Rollenhagen,  “Human  and  Organizational  Biases  Affecting  the  Management  of 
Safety.” 
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Table  2.  Consequences  for  Safety  of  the  Biases  in  the  Four  Thematic  Areas 
Concerning  the  Different  Elements  of  Safety  Management  120 


SAFETY 

MANAGEMENT 

Human 

behavior 

Organizational 

behavior 

Information  and 
uncertainty 

Safety 

models 

Safety  Policy 

BIAS  OF  REASONING: 
Expressed  safety  policies 
focus  on  most  acute  things 
since  these  are  most 
salient,  with  “chronic” 
issues  under-emphasized. 

CAUSAL  PRIMACY 
OF  STRUCTURE: 
Overreliance  on 
technological  means 
for 

controlling 

organizational 

performance 

UNCERTAINTY: 

The  policy  may 
overemphasize  the  need 
for  confidence  and 
certainty  in  an 
environment  where  total 
certainty  is  impossible 

SAFETY  AS  ABSENCE: 

The  policy  may 
overemphasize  error 
prevention  instead  of  safety 
promotion 

Organizing 

HUMAN  ERROR  BIAS: 
The  starting  point  of 
organizing  is  in  preventing 
errors  and  not  in  creating 
the  context  for  safety 
performance 

CAUSAL  PRIMACY 
OF  STRUCTURE: 
Overreliance  on  formal 
organizational 
structure 

in  controlling  safety 

UNCERTAINTY: 
Uncertainty  management 
may  strive  for  reduction 
of  uncertainty  by 
standardization  and 
formalization  without 
perceiving  the  need  for 
flexibility  and 
adaptability. 

SAFETY  AS  ABSENCE: 
Relying  on  removing  the 
negative  may  not  reveal  the 
underlying  dynamics  of  the 
system  that  are  essential  for 
creating  safety 

Risk  analysis 

HUMAN  ERROR  BIAS: 
Human  influence  on  safety 
is  considered  in  negative 
terms  only  as  potential 
errors  that 

threaten  the  otherwise  safe 
system 

CAUSAL  PRIMACY 
OF  STRUCTURE: 
Organizational 
phenomena  such  as 
norms  and  practices 
are  not  considered 
when  evaluating  the 
safety  of 
the  organization 

UNCERTAINTY:  The 
estimatedprobabilities 
become  treated  like 
objective  attributes  of  the 
environment  due  to 
misunderstanding  of 
uncertainty 

SAFETY  AS  ABSENCE: 
Risk  analyses  may  treat  the 
absence  of  failures  as  a 
proof  of  safety 

Experience 

Feedback 

ATTRIBUTION 

ERROR:  Event 
investigations  seek  to 
identify  who  made  what 
mistake  and  attribute 
blame,  instead  of 
identifying  how  the  actors 
made  sense  of  the  situation 
back  then 

NEGLECT  OF 
EMERGENCE: 

Social  phenomena, 
such  as 

norms  and  climate, 
that  can  contribute  to 
both  similar  and 
different  events  in 
future  are 

ignored  in  analyses 

OVER¬ 

QUANTIFICATION: 

Only  quantitative  data  is 
considered  as  valid  for 
risk  analysis,  neglecting 
harder  to  measure  human 
and  organizational  issues 

LINEAR  CAUSALITY; 

Ca  n  lead  to  focus  on  either 
technical,  human  or 
organizational  factors 
as  having  caused  the  event 
under  investigation 

Continuous 

Development 

ATTRIBUTION 

ERROR: 

Development  activities 
might  be 

targeted  at  changing 
individual  behavior  instead 
of  the  wider 
organization  due  to  this 
bias 

LEARNING  AND 
CHANGE:  A  static 
view  on  organization 
ignores  the  gradual 
change  and  optimizing 
of  practices  taking 
place  continuously  in 
the  organization. 

UNCERTAINTY:  A 
quest  for  having  all  the 
“facts”  before  making 
decisions  or 
improvements  may  lead 
to  an  organizational 
inability  to  take  action 

UNDER¬ 

SPECIFICATION: 

Lack  of  specification  on 
what  type  of  safety  (e.g., 
process  or  personnel)  an 
improvement  focuses  on  can 
lead  to  wrong  conclusions 

Safety  Indicators 

BIAS  OF  REASONING; 
Indicator  data  can  be  used 
to  justify  preconceptions 
and 

expectations  that  the 
analyst  had  before  seeing 
the  data,  due 
to  a  confirmation  bias 

CAUSAL  PRIMACY 
OF 

STRUCTURE; 
Indicators  focus  on 
structural  elements 
such  as  technical 
reliability, 

management  system, 
or  instructions, 
neglecting  e.g.,  social 
issues 

OVER¬ 

QUANTIFICATION: 

Only  numerical  indicators 
are  considered  as  valid, 
missing  other  information 
such  as  intuitive 
judgments,  social  norms 
and  subjective  worries. 

SAFETY  AS  ABSENCE: 
Indicators  may  be  selected 
to 

measure  negative 
outcomes(absence)  instead 
of  proactive 

safety  activities  (presence) 

120  Ibid. 
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E, 


DISASTER  SPECULATION 


A  2003  Pentagon  research  project  called  the  Policy  Analysis  Market  (PAM)  was 
billed  as  “betting  on  terrorism.”i2i  While  the  idea  was  so  unpopular  that  the  project  was 
quickly  cancelled,  the  idea  is  not  without  its  merit.  In  designing  a  speculative  market  that 
would  assist  in  making  more  specific  forecasts  regarding  terrorist  attacks,  one  of  the 
developers  of  PAM  described  it  as  “a  research  project  to  test  a  new  forecasting  institution 
being  considered  as  an  addition  to  existing  intelligence  institutions,  to  help  combine 
individual  insights  into  a  consensus  forecast.”  122 

The  aim  of  PAM  was  not  to  predict  terrorist  attacks  but  more  to  focus  on  their 
impact  on  geopolitical  issues  and  destabilizing  effects  throughout  the  world  that  could  be 
influenced  by  terrorist  activities.  Speculative  markets  have  always  used  intelligence  to 
achieve  higher  than  average  profits.  ^23  xhis  intelligence  is  the  product  of  both 
information  and  subjective  opinion.  124  jn  this  case,  the  analytical  opinion  of  one  capable 
of  understanding  the  evolution  of  terrorist  groups  leading  into  future  events. 

PAM’s  unpopularity  came  from  the  prospective  profit  available  through  terrorism 
speculation.  The  fear  was  a  terrorism-dependent  market  that  could  be  manipulated,  to  the 
nation’s  detriment,  by  terrorists.  The  politically  unpopular  project  seemed  to  cross  “a 
moral  boundary,”  125  and  despite  the  favorable  review  it  received  by  those  knowledgeable 
of  its  true  intentions,  12^  the  plan  was  finished.  While  the  possibility  of  financial  gain 
available  from  wagering  on  the  actions  of  terrorists  may  appear  repugnant,  profit  can  take 
on  many  shapes.  If  the  profit  potential  was  presented  as  lives  and  property  saved  from  the 


121  Robin  D.  Hanson,  “Designing  Real  Terrorism  Futures,”  Public  Choice  128,  no.  1/2  (July  2006): 
257-274. 

122  Ibid. 

123  Eric  Zitzewitz  and  Justin  Wolfers,  “Prediction  Markets,”  The  Journal  of  Economic  Perspectives 
18,  no.  2  (Spring  2004):  107-126. 

124  Stephen  Figlewski,  “Subjective  Information  in  a  Betting  Market,”  Journal  of  Political  Economy 
87,  no.  1  (1979):  75-88. 

125  Hanson,  “Designing  Real  Terrorism  Futures.” 

126  Robin  Hanson,  The  Informed  Press  Favored  the  Policy  Analysis  Market  /Fairfax,  VA:  George 
Mason  University,  2005) 
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speculative  analysis  of  potential  terrorist  activities,  one  could  have  anticipated  a  more 
positive  reaction  toward  PAM. 

The  inability  to  collaborate  in  the  name  of  disaster  preparedness  has  a  long 
history.  Agencies,  corporations  and  individuals  often  feel  safer  going  it  alone  or 
continuing  to  do  what  has  already  proven  not  to  work  simply  because  it  has  always  been 
done  that  way.  When  faced  with  a  chaotic  situation,  people  often  revert  back  to  a  comfort 
system  that  they  are  familiar  with.  Organizations  need  to  prepare  under  simple  or  known 
conditions  in  order  to  be  able  to  function  effectively  under  complex  or  chaotic  situations 
(See  Fig  3).  Peter  Wilkin  references  this  idea  when  he  states  “the  relationship  between 
the  necessary  (structural  relations)  and  the  contingent  (the  acts  of  agents  as  groups  or 
individuals)  disappears.”  1^7  in  a  desire  to  achieve  true  community  resiliency, 
organizations  must  first  see  themselves  as  members  of  the  community  and  as  individual 
agencies  or  businesses  secondly.  Organizations  need  to  avoid  preconceived  notions  that 
will  compromise  their  emergency  management  capabilities,  the  safety  of  employees  and 
their  overall  contribution  to  a  resilient  community. 


complex 
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Figures.  The  Cynefin  Framework 


This  chapter  has  attempted  to  look  at  various  social  aspects  that  affect  a  mindset 
toward  or  away  from  collaboration  and  experience  contributions  from  outside  sources.  In 


127  Wilkin,  “The  Ideology  of  Ergonomics. 
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an  effort  to  increase  safety  and  preparedness,  and,  through  collaborative  efforts,  minimize 
organization  deviances  for  public  and  private  stakeholders. 

Collaboration  alone  will  not  overcome  the  roadblocks.  Valuable  lessons  may  be 
missed  because  of  the  bias  held  for  one’s  own  organization  or  community.  128  Agencies, 
public  and  private,  need  to  continually  develop  cooperative  relationships  and  prevent  the 
dissolving  of  existing  ones.  Without  nurturing,  the  benefits  of  collaboration  wither. 


128  Enrico  L.  Quarantelli,  Human  Resources  and  Organizational  Behaviors  in  Community  Disasters 
and  Their  Relationship  to  Planning  (Newark,  DE:  Disaster  Research  Center,  1982). 
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IV.  ANALYSIS  OF  THE  AFTER  ACTION  REPORTS 


Practice  does  not  make  perfect.  Only  perfect  practice  makes  perfect. 

— Vince  Lombardi, 
American  football  player  and  eoaeh 


A,  METHODOLOGY 

This  thesis  requires  an  evaluation  of  current  exereise  design  value  and  the 
development  of  the  Intelligent  Exereise  Model.  The  culmination  is  a  greater  collaboration 
between  fire  service  intelligence  entities  and  the  exercise  design  process.  Specifieally 
designed  for  the  FDNY’s  Center  for  Terrorism  and  Disaster  Preparedness  (CTDP),  the 
model  will  serve  as  a  visual  meehanism  to  reinforce  the  need  for  intelligence 
contributions  to  the  exercise  design  proeess  applieable  to  any  municipal  agency. 

To  support  thesis  research,  After  Aetion  Reports  (AAR)  from  numerous  events, 
exercises  and  ineidents  were  collected  and  analyzed.  The  purpose  was  to  conduct  a 
content  analysis,  coding  for  themes  and  significant  language  that  allows  us  to  traek 
correlation  between  aetual  response  issues  and  exereise  issues.  These  AARs  reviewed 
were  taken  from  exereises  and  aetual  responses  to  emergencies  from  within  the  United 
States  that  represent  a  comprehensive  sampling  of  the  evaluative  proeess  and  its  effeet  on 
improvement  planning.  This  thesis  is  not  focused  on  proximity  of  exercises  and  events. 
Since  the  focus  is  on  shared  intelligence  aeross  boundaries,  location  was  not  considered 
to  be  a  limiting  factor.  While  it  will  be  expected  that  agencies  are  more  likely  to  benefit 
from  lessons  learned  proximal  to  their  area  of  responsibility,  ageneies  have  a 
responsibility  to  seek  out  intelligence  gathered  from  exereises  and  events  and  apply  that 
to  their  jurisdiction. 
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B, 


CONTENT  ANALYSIS 


Richard  Boyatzis  stated  in  Transforming  Qualitative  Information:  Thematic 
Analysis  and  Code  Developmenf^^  that  the  four  stages  in  developing  the  ability  to  use 
thematic  analysis  are: 

•  Sensing  Themes — recognizing  the  eodable  moment 

•  Doing  it  Reliably — reeognizing  the  eodable  moment  and  eneoding  it 
eonsistently 

•  Developing  Codes 

•  Interpreting  the  Information  and  Themes  in  the  Context  of  a  theory  or 

Conceptual  Framework — eontributing  to  the  development  of  knowledge 

There  are  three  distinet  stages  involved  in  the  use  of  thematic  analysis:  Stage  I, 
deciding  on  sampling  and  design  issues;  Stage  II  developing  themes  and  a  code  and  stage 
III  is  where  we  begin  validating  and  using  the  eode. 

Stage  II  is  further  divided  into  three  different  ways  to  develop  a  thematie  code: 

•  Theory  driven 

•  Prior  data  or  prior  research  driven 

1  -5  1 

•  Induetive  data  driven 

In  Stage  I,  the  sample  of  101  AARs  was  chosen  to  be  representative  of  the  AAR 
following  the  HSEEP  framework.  This  would  reveal  trends  that  eould  be  graphieally 
illustrated  for  demonstration  purposes. 

In  Stage  II,  the  themes  were  revealed  as  the  data  were  colleeted.  Initially  the 
eategories  used  for  eoding  the  recommendations  started  at  zero,  sinee  categories  were  not 
predetermined. 

Content  analysis  can  be  defined  as  “a  researeh  technique  for  making  replicable 

1  T9 

and  valid  inferenees  from  data  to  their  eontext.”  The  intent  here  was  to  synthesize  the 

129  Richard  E.  Boyatzis,  Transforming  Qualitative  Information:  Thematic  Analysis  and  Code 
Development  (Thousand  Oaks,  CA:  SAGE,  1998). 

130  Ibid.,  11. 

131  Ibid.,  29. 

132  Krippendorff,  Content  Analysis .  31. 
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recurring  recommendations  from  AARs  for  real  world  events  and  compare  them  to 
objectives  established  for  exercises.  Recommendations  from  exercises  were  collected  as 
well.  This  data  was  then  analyzed  and  compared.  The  four  key  components  of  Content 

1  TT 

Analysis  are  measurement,  indication,  representation  and  interpretation. 

1.  Measurement 

The  measurement  procedures  of  content  analysis  are  based  on  frequency  counts  of 
semantically  equivalent  textual  units  such  as  words,  word  senses  phrase  or  themes.  ^34 
The  data  collected  represented  the  frequency  with  which  certain  words  or  phrases 
appeared  among  the  recommendations  in  each  AAR.  These  were  coded  and  sorted  into 
one  of,  what  eventually  became  104  categories.  The  categories  were  chosen  simply 
because  they  existed  among  the  recommendations.  Only  one  of  the  real  world  AARs 
included  a  recommendation  that  addressed  a  Special  Weapons  and  Tactics  (SWAT)  team 
and  only  two  included  recommendations  concerning  the  Stafford  Act.  Both  were  included 
as  part  of  the  data  set  for  two  reasons.  One  was  to  maintain  objectivity.  It  was  important 
not  to  omit  any  entries  not  matter  how  seemingly  insignificant  they  may  have  appeared  in 
order  to  create  an  accurate  sample.  Secondly,  while  the  SWAT  reference  may  have 
appeared  statistically  insignificant  initially,  its  comparative  value  when  paired  against 
recommendations  from  exercise  AARs  becomes  more  evident.  It  was  impossible  to  tell 
this  during  the  collection  process  so  all  AAR  recommendations  were  included. 

2.  Indication 

Indication  refers  to  the  unmeasured  aspects  of  the  text  as  inferred  by  the  analyst. 
For  example,  if  an  AAR  refereed  repeatedly  to  a  recommendation,  it  would  only  be 
counted  where  it  belongs.  Special  inclusion  for  overstated  recommendation  words  such  as 
must,  never  or  always  or  for  understated  words  such  as  may,  could  or  some  were  not 
given  as  the  objectivity  of  the  person  writing  the  AAR  was  not  viewed  as  important  as  the 
content.  135  One  AAR  included  obvious  bias  toward  the  issues  affecting  the  special  needs 


133  Weber,  Basic  Content  Analysis.lt). 

134  Ibid.,  72. 

135  Ibid.,  75. 
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community  and  nearly  every  reeommendation  had  speeial  needs  concerns,  though  the 
objeetives  of  that  exercise  were  not  specifically  geared  toward  a  special  needs  population. 
The  recommendations  were  interpreted  and  categorized  as  needed  for  “speeialized 
equipment,”  “transportation  resiliency”  and  “social  media,”  but  eaeh  reeommendation 
was  categorized  based  on  its  content  and  the  “Special  Needs”  category  received  its  single 
checkmark. 


3.  Representation 

Often  eritieized  for  its  language  coding  limitations,  content  analysis  ean  have  the 
appearance  of  maintaining  certain  linguistie  biases.  136  Since  the  interpretation  of  data 
into  eoded  analysis  is  the  responsibility  of  the  analyst,  it  is  important  to  elassify  words  or 
phrases  by  the  references  they  make.  That  is,  to  ignore  the  subjective  preferences  of  the 
ereator  of  the  AAR  and  instead  concentrate  on  the  material  reeommendations.  The 
intended  definition  was  drawn  by  the  comprehensive  AAR  and  the  analyst’s  familiarity 
with  the  proeess.  Any  recommendations  that  were  misinterpreted  from  the  intention  of 
the  AAR  author(s)  was  unintentional  though  would  have  been  consistent  throughout  the 
information  gathering  proeess  for  this  thesis.  To  summarize,  though  there  may  be  more 
valuable  text  analysis  systems,  some  distinctions  in  language  were  not  considered  worth 
maintaining.  137 

4.  Interpretation 

Each  recommendation  needed  to  be  interpreted  in  order  to  find  its  needed 
location.  The  data  did  not  require  a  high  level  of  interpretive  eomplexity.  138  Speeifically, 
if  a  recommendation  was  made  that  included  an  inability  to  reeognize  the  Incident 
Commander  in  the  EOC,  then  that  was  included  as  a  reeommendation  in  the  EOC,  ICS 
and  ICS  identification  categories.  While  this  would  leave  some  AARs  being  represented 
in  more  eategories  than  may  have  seemed  necessary,  it  ensured  complete  inclusion  and 


136 

137 

138 


Krippendorff,  Content  Analysis,  43. 

Weber,  Basic  Content  Analysis,  77. 

Vaismoradi,  Turunen,  and  Bondas,  “Content  Analysis  and  Thematic  Analysis,”  399. 
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also  minimized  interpretation  bias  that  would’ve  resulted  in  using  less  categories  based 
on  the  analyst’s  familiarity  with  the  AAR  process. 

C.  THE  AFTER  ACTION  REPORTS 

1.  Selection  of  the  AARs 

The  AARs  selected  for  review  were  picked  to  cover  a  variety  of  emergency 
circumstances,  which  would  be  relevant  to  as  many  scenarios  as  possible.  For  example, 
the  issues  that  face  communities  and  responders  during  a  hurricane  have  similarities  to 
those  that  occur  after  tornados  or  blackouts.  The  effects  of  these  emergencies  on  the 
healthcare  system  are  similar  to  those  that  can  be  expected  during  a  medical  surge  for  a 
variety  of  scenarios.  Therefore,  the  goal  is  to  evaluate  a  large  enough  sample  of 
emergencies  to  include  as  many  of  the  core  capabilities  as  possible.  Significant  events 
were  included  that  were  easily  recognizable  (i.e.,  the  Oklahoma  City  bombing.  Hurricane 
Katrina  or  the  School  Shooting  at  Columbine).  Then,  by  comparing  observations 
uncovered  during  emergencies  with  objectives  established  during  the  exercise  design 
process,  either  a  clear  communal  understanding  of  lessons  learned  will  emerge  or  a  gap 
will  surface  indicating  a  need  for  greater  collaboration  between  intelligence  and  exercise. 

In  order  to  determine  whether  the  hypothesis  is  correct,  that  we  are  not 
incorporating  lessons  learned  from  actual  events  to  exercises  conducted  afterward,  AARs 
were  selected  where  a  similar  exercise  occurred  subsequent  to  the  event.  Wherever 
possible,  data  from  an  exercise  that  was  conducted  before  the  actual  event  would  be 
included  to  provide  a  metric  of  beneficial  learning.  Events  were  chosen  where  the  AAR 
was  developed  following  the  format  set  forth  in  the  Homeland  Security  Exercise  and 
Evaluation  Program  (HSEEP).  The  same  was  done  for  the  exercise  AARs.  This  would 
lend  consistency  to  the  sampling  and  make  the  extraction  of  recommendations  and 
objectives 

It  is  believed  that  these  exercises  did  not  necessarily  need  to  be  restricted 
geographically.  Because  of  the  availability  of  lessons  learned  platforms,  exercises 
conducted  in  California  are  available  to  benefit  exercise  planners  in  New  York.  Eikewise, 
the  lessons  learned  from  a  hurricane  occurring  on  the  North  Carolina  Coast  should 
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provide  benefit  to  responders  and  emergeney  managers  in  Texas.  Therefore, 
ineonsisteneies  notieed  between  the  AARs  eompiled  after  events  and  exereises  are  being 
evaluated  on  a  systemie  basis.  That  is,  whether  exereises  are  eonducted  as  a  direct  result 
of  a  local  emergency  (Tornado  exercise  conducted  in  Joplin,  MO  or  a  Hurricane  exercise 
in  NY  City).  A  gap  in  this  area  should  strengthen  the  need  for  increased  collaborative 
intelligence  sharing.  As  common  lessons  learned  emerge  during  emergencies,  we  should, 
ideally,  see  these  identified  as  objectives  in  future  exercises.  It  is  the  hypothesis  of  this 
thesis,  however,  that  objectives  and  lessons  learned  will  be  dissimilar. 

The  sample  size  was  drawn  from  exercises  and  events  around  the  country 
adhering  to  the  following  guidelines: 

•  Only  AARs  for  events  occurring  within  the  United  States  were  included 

•  AARs  had  to  follow  the  HSEEP  format  - 

•  The  range  of  AARs  was  from  2001  through  the  present  -  The  goal  was  to 
focus  primarily  on  post  9/11  recommendation-heavy  climate  adhering  to 
modern  formats.  ^39 

•  Approximate  even  distribution  was  maintained  between  exercises  and  real 
world  events  to  maintain  parity.  (53  exercises  and  48  real  World  events) 

Searches  for  AARs  were  conducted  on  the  DHS  Eessons  Teamed  Information 
Sharing  (EEIS)  website  and  The  Homeland  Security  Digital  Eibrary  (HSDE)  and 
additional  Google  searches.  The  Google  searches  yielded  several  collections  of  AARs  but 
one  of  the  most  valuable  was  on  the  WisconsinWatch.org  website.  The  website  was 
produced  by  the  Wisconsin  Center  for  Investigative  Journalism  who  requested  AARs 
from  counties  across  Wisconsin.  The  reports  range  from  2005-2011  and  were  written  by 
a  variety  of  agencies,  This  accounts  for  the  disproportionate  number  of  Wisconsin 


139  7he  only  exceptions  were  the  1999  Columbine  School  shooting  and  the  1995  Oklahoma  City 
bombing  because  of  their  significance  as  landmark  events  in  homeland  security,  even  though  they  occurred 
before  the  notion  of  “Homeland  Security”  came  into  existence.  They  were  included  because  of  their  history 
and  because  the  reports  available  were  deemed  sufficiently  formatted  in  such  a  way  that  recommendations 
could  be  easily  extracted.  While  other  pre-9/11  events  could  also  be  viewed  as  seminal,  e.g.,  the  FBI 
standoffs  at  Ruby  Ridge  and  Waco,  Hurricane  Andrew  or  the  crash  of  TWA  Flight  800,  the  primary  focus 
was  to  be  post  9/11. 

140  They  Learned  in  the  Disaster:  Agencies’  After-Action  Reports,”  WisconsinWatch.org, 
accessed  January  14,  2014,  http://www.wisconsinwatch.org/viz/what-they-learned-in-the-disaster-agencies- 
after-action-reports/. 

141  Ibid. 
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AARs  represented  in  the  database.  Whether  some  states  exercise  more  than  others, 
produce  more  consistent  AARs  after  exercises  or  emergencies,  or  simply  made  their 
AARs  more  accessible  was  unclear. 

New  York  City  and  Washington,  DC,  were  singled  out  as  sole  city  representatives 
because  of  the  September  1 1  attacks.  Figure  4  shows  the  AAR  contribution  by  state  for 
both  exercise  and  real  world  events. 


■  Exercises  ■  Real  World  Events 

Figure  4.  Exercise  Contribution  by  State 

Once  found,  the  AARs  were  briefly  skimmed,  first  for  U.S.  origin  then  HSEEP 
compliance.  They  were  then  filed  simply  as  an  exercise  or  real  world  event.  It  was 
important  not  to  read  the  AARs  at  this  point  in  order  to  maintain  the  objectivity  of  the 
sample. 


2,  Coding  of  the  AARs 

Content  and  Thematic  Analysis  allow  the  analyst  to  remain  objective  when 
collecting  data.  Either  the  information  occurs  or  it  does  not.  Since  the  AARs  are  all 
historical  documents,  meaning  the  information  entered  into  them  was  done  so  before  and 
separate  from  the  purposes  of  this  paper,  there  is  no  interaction  between  data  submission 


41 


data  collection  which  could  bias  the  findings.  Content  analysis  helps  those  evaluating  to 
leam  more  about  the  data  they  are  examining  because  it  is  systematic, 

The  coding  of  the  AARs  was  based  on  the  frequency  of  common  exercise  and 
emergency  issues,  for  example,  ICS,  communications,  patient  management,  etc.  The 
occurrences  of  common  phrases  will  be  gathered  and  tracked  and  trends  among  AARs 
will  be  included.  The  information  was  then  assigned  to  categorical  variables 
distinguishing  among  subject,  timing  and  event  and  classified  into  finite  number  of 

categories.  143 

The  recommendation  section  of  each  AAR  was  coded  for  themes  in  the  areas  of 
improvement  across  exercises  and  real  world  events.  As  a  new  recommendation  was 
discovered  a  new  category  was  created.  This  eventually  resulted  in  104  different 
categories  The  codification  of  the  AARs  supplied  a  comparative  metric  whereby  the 
information  contained  in  the  recommendations  could  be  assigned  a  numeric  value  and 
consistently  measured  against  the  remaining  information  either  wholly  or  partially. 

3,  Analysis  of  the  AAR  Coding 

The  occurrence  rate  among  emergency  AARs  was  compared  to  the  frequency  of 
inclusion  during  exercises.  The  expectation  is  this:  if  the  AAR  following  an  emergency 
identifies  a  shortcoming,  this  gap  should  be  addressed  in  future  exercises. 

As  emergencies  occur  across  the  nation,  the  frequency  of  issues  raised  exhibits 
consistency  from  region  to  region;  therefore,  the  lessons  learned  should  similarly 
maintain  a  consistency  that  is  translated  to  exercise  development  regardless  of  location. 

The  data  were  collated  into  over  100  classifications.  The  first  category  that  was 
tracked  was  the  amount  of  warning  time  provided  to  civilians  and  responders  during  the 
scenario,  whether  exercise  or  real.  The  scenarios  were  divided  among:  No  warning, 
minimal  warning  and  significant  warning  events.  No  warning  events  were  those  where  no 
realistic  warning  could  be  expected,  such  as  an  earthquake,  terrorist  attack  or  train 
derailment.  The  third  category  was  for  scenarios  where  significant  warning  was  available, 

142  United  States  General  Accounting  Office,  Content  Analysis:  A  Methodology  for  Structuring  and 
Analyzing  Written  Material  (Washington,  DC:  U.S.  General  Accounting  Office,  1996), 

http ://  WWW.  gao .  go  v/  products/PEMD  -10.3.1. 

143  Ibid. 
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scenarios  such  as  hurricanes  or  winter  storms  that  could  be  tracked  for  days  or  even 
weeks  prior  to  impact.  Everything  else  was  considered  a  minimal  warning  event.  Those 
which  some  amount  of  warning  would  be  available  like  tornadoes  or  wild  fires.  These 
were  listed  as  minimal  warnings  because  there  is  a  predictable  season  and  while  we 
cannot  calculate  where  and  when  either  will  strike;  there  is  some  leverage  of 
preparedness  afforded.  Specifically,  they  did  not  fall  into  either  the  No  Warning  or 
Significant  Warning  categories.  The  second  category  tracked  the  variety  of  hazards 
associated  with  the  AARs.  This  would  be  used  to  determine  whether  exercises  were  being 
conducted  in  accordance  with  real  world  events.  The  14  categories  ranged  from  events 
associated  with  criminal  activity  and  terrorism  to  events  caused  by  severe  weather  and 
natural  hazards  like  ice  storms  and  earthquakes.  The  hypothesis  was  that  there  would  be 
inconsistencies  with  the  frequency  of  events  that  actually  occurred  and  those  that  served 
as  an  exercise  scenario. 

The  third  and  largest  category  was  for  recommendations.  There  were  104  separate 
classifications  of  recommendations  taken  from  all  of  the  AARs.  These  were 
recommendations  from  both  exercises  and  real  world  events  and  were  sorted 
independently. 

The  last  category  was  reserved  for  exercise  objectives.  These  71  objectives  were 
taken  from  the  exercise  AARs  and  were  coordinated  for  similarities  among  the 
recommendations  in  order  to  clearly  associate  the  recommendations  with  the  objectives 
when  the  analysis  of  the  data  was  conducted.  The  thought  here  was  that  exercise 
objectives  would  not  correlate  to  recommendations  and  would  further  the  hypothesis  that 
there  is  a  need  for  the  insertion  of  intelligence  into  the  exercise  design  process.  When 
reviewing  the  first  category  of  AARs,  the  results  (see  Figure  5)  show  that  exercises 
favored  “no  warning”  scenarios  over  “significant  warning”  scenarios  by  a  nine  to  one 
ratio  (54%  of  exercises  featured  scenarios  that  were  without  warning  while  only  6% 
focused  on  scenarios  with  significant  warning  periods.) 

Real-world  AARs  showed  almost  an  even  distribution  between  “no  warning”  and 
“significant  warning.”  This  is  indicative  of  an  inconsistency  in  exercise  planning.  If  the 
emergency  community  responds  more  frequently  to  events  where  they  have  significant 
warning  then  should  the  exercises  they  train  on  be  developed  in  a  similar  manner. 
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Looking  at  the  scenarios  that  were  extracted  from  the  database  of  AARs,  there  are  two 
noticeable  variances  (see  Figure  6).  Real-world  events  featured  incidents  of  severe 
weather  42%  of  the  time  compared  to  only  25%  as  an  exercise  scenario  marking  a  71% 
increase  from  exercise  to  real  world  for  that  scenario.  Similarly,  terrorism  was  featured  in 
23%  of  the  exercises  but  only  6  %  of  the  real  world  AARs.  This  represents  a  staggering 
262%  increase  from  real  world  to  exercises. 


No  Miniinal  Significant 

Warning  Warning  v^/arning 


■  Exercises  ■  Real  World  Events 


Figure  5.  Warning  prior  to  Hazard 
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■  Exercises  ■  Real  World  Events 

Figure  6.  Scenario  Distribution 


While  terrorism-related  scenarios  may  be  popular  their  occurrence  in  real-world 
events  was  minimal  and  should  be  reflected  proportionately  in  exercise  scenarios.  A  more 
accurate  use  of  the  terrorist  scenario  is  reflected  if  we  look  at  exercises  conducted  after 
2008,  though.  In  the  AARs  reviewed,  prior  to  2008,  terrorism  was  used  as  an  exercise 
scenario  nine  times  as  opposed  to  only  three  times  since.  This  is  consistent  with  its 
appearance  in  the  real  world  AARs  and  is  representative  of  the  shift  from  preparedness 
for  terrorism  events  to  a  more  all-hazards  approach  in  exercise  planning.  1^4  Figure  7  is 
representative  of  the  shift  in  AARs  corresponding  with  the  shift  in  preparedness  scenarios 
as  the  data  was  analyzed  pre  and  post  2008. 


Department  of  Homeland  Security,  Quadrennial  Homeland  Security  Review  Report. 
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■  Exercises  ■  Real  World  events 

Figure  7.  Terrorism  Scenarios  Pre-  and  Post-2008 
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■  Real  World  Events  ■  Exercises 


Figure  8.  Recommendations  Sorted  by  Real-World  Events 


When  the  recommendations  were  compared  between  exercises  and  real-world 
events  the  results  were  not  surprising.  Sorting  the  results  by  both  real  world  events  and 
exercises  yielded  similarities  among  needs.  Among  the  top  five  in  both  categories  were 
communications,  training  and  exercise,  problems  with  the  Incident  Command  System 
(ICS),  resource  management  and  issues  in  the  Emergency  Operations  Center  (EOC). 
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Figure  8  lists  the  top  20  oeeurring  reeommendations  among  Real  World  AARs  and 
Exereise  AARs. 

4,  Objective  Recommendations 

The  similarities  among  reeommendations  are  the  standard  issues  that  the  exereise 
design  eommunity  is  familiar  with  and  these  findings  should  eome  as  no  surprise.  It 
should  reason,  then,  that  exereise  objeetives  should  show  a  similar  concentration  of 
issues.  If  we  repeatedly  expose  the  same  areas  for  improvement  in  our  AARs  then  these 
would  likely  be  the  areas  that  receive  the  most  attention  when  it  comes  to  our  exercises. 
Figure  9  shows  a  somewhat  different  finding,  however.  Based  on  the  content  analysis 
conducted  over  the  sample  for  this  thesis,  there  were  some  significant  differences. 

When  looking  at  the  top  five  based  on  disparity  between  recommendation  and 
objective,  resource  management,  documentation,  multi  jurisdictional  issues,  the  public 
information  officer  (PIO)  and  mutual  aid  and  memorandums  of  understanding  (MOUs), 
there  is  a  noticeable  divide  between  each.  Documentation,  for  example,  was  an  issue  in 
48%  of  the  real  world  AARs  yet  only  found  inclusion  as  an  objective  in  4%  of  exercises. 
Improperly  documented  associated  costs  are  the  primary  reason  local  governments  fail  to 
receive  reimbursement  after  a  disaster.  1^5  So  by  addressing  this  one  objective  or,  at  a 
minimum,  closing  the  gap,  localities  would  be  better  able  to  qualify  for  reimbursement. 
More  reimbursement  would  actually  allow  them  to  fund  additional  exercises. 


State  of  Indiana,  “Reimbursement  Procedure  for  FEMA  Public  Assistance,”  IN.gov,  March  2006, 
http://www.in.gov. 
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■  Exercise  Objectis'es  ■  Real  World  Recommendations 

Figure  9.  Disparity  between  Real-World  Reeommendations  and  Exereise 

Objeetives 


There  were  also  reeommendations  that  appeared  frequently  and  were  eompletely 
ignored  as  exereise  objeetives.  Figure  10  demonstrates  the  reeommendations  that 
reeeived  no  inelusion  as  objeetives.  This  was  a  signifieant  finding  as  the  frequeney  of 
these  reeommendations  was  not  minimal. 


■  Exercise  Objectives  ■  Real  World  Recommendations 


Figure  10.  Reeommendations  Reeeiving  Zero  Inelusion  as  Objeetives 
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D,  ANALYSIS  OF  THE  FINDINGS 

It  is  apparent  from  the  analysis  that  reeommendations  from  exercises  and  real 
world  events  are  not  being  transferred  into  our  exercise  process.  This  is  a  problem.  What 
this  means  is  that  the  emergency  management  community  is  content  to  keep  exercising 
the  wrong  issues,  not  exercising  consistent  problems  and  not  learning  from  the 
intelligence  gathered  and  analyzed  after  exercises  and  real  world  disasters.  This 
intelligence  should  be  the  driving  force  to  guide  improvement  and  is  being  largely 
ignored.  We  seem  to  be  carrying  a  predetermination  toward  our  future  exercises.  A 
predetermination  that  suggests  recommendations  almost  pre-scripted. 

E.  LIMITATIONS  OF  THE  METHOD 

It  is  understood  that  this  method  contains  certain  limitations.  Listed  below  are  the 
most  obvious  with  an  accompanying  analysis  and  explanation. 

1.  Sample  Size 

The  sample  size  of  101  AARs  represents  only  a  small  portion  of  exercises  and 
real  world  events.  Sample  size  will  always  affect  the  data  produced.  100  events  will 
produce  a  more  accurate  pool  of  data  than  a  sample  of  10  but  far  less  than  a  pool  of 
1,000.  The  sample  includes  a  mix  of  AARs  from  30  states,  which  left  out  many  locations. 
This  was  primarily  due  to  the  readily  accessible  nature  of  the  chosen  AARs.  Specific 
AARs  were  not  sought  out  with  some  exceptions.  September  1 1  AARs  were  chosen  as 
was  the  DC  sniper  AAR.  The  Oklahoma  City  bombing  from  1995,  the  Columbine  school 
shooting  from  1999  were  included  as  were  select  AARs  from  Hurricanes  Katrina  and 
Sandy  in  order  not  to  neglect  landmark  events. 

2.  Subjectivity  in  Selection 

One  of  the  concerns  when  selecting  the  AARs  for  the  data  sample  was  the 
objectivity  of  the  process.  As  explained  earlier,  AARs  were  chosen  simply  because  of 
HSEEP  compliance  and  U.S.  origin.  The  intent  was  to  maintain  as  much  consistency  in 
format  and  randomness  in  selection,  though  fully  achieving  this  may  not  have  been 
possible. 
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3. 


Distribution 


Several  states  were  disproportionately  represented  in  the  sample  and  some  states 
were  not  represented  at  all.  While  this  was  a  result  of  the  aeeessibility  of  the  AARs  and 
not  a  preferenee  for  any  region,  the  unevenness  of  distribution  may  be  ineorreetly 
attributed  to  bias. 

4.  Limitation  Solutions 

The  limitations  explained  above  eould  be  addressed  by  examining  a  larger  sample 
of  After  Aetion  Reports.  While  104  may  have  served  the  purpose  of  this  analysis,  a  larger 
eross  section,  including  more  locations,  greater  scenario  inclusion  and  use  of  a  wider 
sample  of  Federal,  State  and  Local  AARs,  would  have  provided  a  more  complex  data  set, 
likely  resulting  in  a  more  accurate  and  complete  analysis. 

Including  more  researchers  would  have  given  greater  subjectivity  to  the  analysis 
of  the  data.  This,  too,  would  have  increased  the  depth  of  the  results  and  reduced  the 
possible  appearance  of  analyst  bias  in  the  findings. 
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V.  THE  CONNECTION  BETWEEN  INTELLIGENCE  AND 

EXERCISE 


It  does  not  do  to  leave  a  live  dragon  out  of  your  calculations,  if  you  live 
near  him. 

— J.R.R.  Tolkien,  The  Hobbit 


A,  INTELLIGENCE-FREE  EXERCISE 

When  exereises  are  conducted  in  a  vacuum,  that  is,  unaffected  by  surrounding 
environmental  influences,  they  provide  only  of  minimal  benefit.  Organizations  train  on 
what  they  are  already  efficient  at  and  do  very  little  to  address  or  even  identify  their 
systemic  gaps.  It  has  been  suggested  that  the  planning  process  is  more  important  than  the 
procedures.  Planning  is  often  a  result  of  the  most  recent  disaster  and  not  based  on  any 
systematic  lessons  learned.  “This  would  pose  no  problem  if  the  common  sense  notions 
and  assumptions  made  about  emergency  time  behavior  were  valid,”  though  social 
scientists  have  recently  asked  some  serious  questions  about  common  expectations  during 
disaster  responses.  1^6  while  there  may  be  some  predictability  of  a  crowd  during  an 
emergency  evacuation,  properly  preparing  a  population  to  respond  during  a  disaster 
situation  will  raise  the  level  of  expected  safety. 

When  we  evaluate  exercises  it  is  useful  to  use  something  similar  to  the  Disaster 
and  Exercise  Performance  Information  Collection  Tool  (DEPICT),  developed  by  a  joint 
civilian  and  military  medical  response  team.  1^8  a  repository  for  data  collected  from 
exercises  or  real  world  disasters,  DEPICT  allows  field  based  observations  to  be  broadly 
collected  then  analyzed  with  minimal  bias.  Erequently,  the  author  of  an  evaluation  is 
someone  who  was  actually  involved  in  the  incident  or  was  responsible  for  some  aspect  of 
the  planning  or  response.  It  is  not  easy  for  one  to  impartially  evaluate  the  actions  of  their 

Quarantelli,  Human  Resources  and  Organizational  Behaviors.’’' 

Peter  W.  Blaich,  The  Benefits  of  Behavioral  Research  to  the  Fire  Service-Human  Behavior  in 
Fires  and  Emergencies  (Lincoln,  NE:  iUniverse,  2008). 
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own  organization.  “Too  often,  post-disaster  eritiques  turn  out  to  be  defenses  or 
justifieations  of  what  was  done,  rather  than  objeetive  assessments  of  problems  and 
mistakes.”  149  By  not  incorporating  an  intelligence  aspect  into  the  planning  of  our 
exercises  and  thus  our  evaluations,  we  may  be  missing  valuable  lessons  “because  of  the 
bias  held  for  one’s  own  community  or  organization.” 

B,  HSEEP  FOR  TERRORISTS 

How  do  terrorists  learn?  Can  we  assume  that  they  have  a  structured  framework 
for  evaluation  that  is  conducted  after  each  attack,  whether  successful  or  not?  By 
comparing  a  string  of  “failed”  attacks  we  can  surmise  a  “lessons  learned”  structure  of 
some  sort.  If  so,  then  we  as  Homeland  Security  professionals  should  be  able  to  further 
learn  from  this  in  an  effort  to  be  prepared  for  an  evolving  terrorist  threat.  As  a  terrorist 
group  learns  it  can  repair  gaps  in  its  structure  and  strengthen  its  capabilities.  An  ability  to 
learn  allows  a  terrorist  group  to  adapt  to  its  changing  surroundings  and  goals  by: 

•  Developing,  improving,  and  employing  new  weapons  or  tactics  that  can 
enable  it  to  adjust  its  capabilities  as  necessary 

•  Improving  its  members’  skills  in  applying  current  weapons  or  tactics 

•  Collecting  and  utilizing  the  intelligence  information  needed  to  mount 
operations  effectively 

•  Thwarting  countermeasures  and  improve  its  chance  of  surviving  attempts 
to  destroy  it 

•  Preserving  the  capabilities  it  has  developed  even  if  some  of  its  members 
are  lost.^-^^ 

Observing  the  organizational  learning  that  takes  place  among  a  group,  more 
specifically  than  from  individual  to  individual,  we  can  begin  to  see  areas  where  we  can 
adapt  our  response  in  a  similar,  though  far  more  positive  manner.  By  understanding  how 
terrorist  groups  continue  to  learn  in  an  ever  changing  arena,  response  agencies  can  better 
organize  to  respond  to  and  prepare  for  inevitable  terrorist  attacks. 


149  Auf  der  Heide,  Disaster  Response. 

1^1*  Quarantelli,  “Human  Resources  and  Organizational  Behaviors.” 
1^1  Jackson  et  at,  Aptitude  for  Destruction. 


52 


In  order  to  benefit  from  the  organizational  learning  exhibited  by  a  terrorist 
organization,  it  is  neeessary  to  understand  why  responders  would  want  to  learn  from  them 
in  the  first  plaee.  The  objectives  of  the  response  organization  need  to  be  clearly 
understood.  If  the  goal  is  preparedness  and  response,  then  understanding  the  terrorist 
intentions  and  tracking  improvement  plans,  those  tasked  with  preparing  to  respond  to  the 
next  terrorist  attack  are  armed  with  preparatory  knowledge. 

The  “Open  Source  Jihad”  section  of  Inspire  Magazine,  described  as  “a  resource 
manual  for  those  who  loathe  tyrants  ...  a  disaster  for  the  repressive  imperialistic  nations; 
the  open  source  jihad  is  America’s  worst  nightmare,”  1^2  provides  easy-to-follow 
suggestions  and  instructions  for  would  be.  Home  Grown  Violent  Extremists  (HGVE).  An 
article  in  the  magazine  posed  this  question:  “Can  I  make  an  effective  bomb  that  causes 
damage  to  the  enemy  from  ingredients  available  in  any  kitchen  in  the  world?  The  answer 
is  yes.”  And  it  went  on  to  describe  an  easily  assembled  explosive  device  in  the  article 
“Make  a  Bomb  in  the  Kitchen  of  Your  MomY^^^  This  was  eventually  the  source  for  the 
explosive  device  used  in  the  2013  bombing  at  the  Boston  Marathon, 

This  is  significant  both  tactically  and  strategically.  Tactically,  it  tells 
counterterrorism  professionals  to  be  stalwart  in  identifying  the  gathering  of  seemingly 
innocuous  materials.  Terrorists  can  create  dangerous  explosives  easily,  but  more 
importantly,  they  can  do  so  without  leaving  the  country  for  expensive  training  that  is 
available  for  security  professionals  to  notice,  By  understanding  not  only  the  methods 
but  the  motivation  behind  the  terrorist  intentions,  we  can  begin  to  comprehend  them 
strategically. 

It  becomes  possible  to  anticipate  the  terrorists’  next  logical  step  by  understanding 
their  strategy.  Understanding  their  particular  goals  and  objectives  and  the  tradeoffs  they 


152  Qaida’s  English-Language  Terrorism  Blueprint:  The  Investigative  Project  on  Terrorism,” 
accessed  February  18,  2014,  http://www.investigativeproject.org/2056/al-qaidas-english-language- 
terrori  sm-b  lueprint . 

1^3  The  AQ  Chef,  “Make  a  Bomb  in  the  Kitchen  of  Your  Mom,”  Inspire,  Summer  2010. 
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will  be  willing  to  make  in  order  to  aecomplish  these,  allow  counterterrorism  professionals 
comprehend  when  available  options  have  been  minimized, 

Familiarity  with  the  learning  processes  of  terrorist  groups  could  make  it  possible 
to  develop  countermeasures  to  combat  or  possibly  take  advantage  of  terrorist  groups’ 
attempts  to  learn.  Attacking  the  group’s  “learning  systems”  may  lessen  its  ability  to 
evolve  over  time,  While  the  advances  made  by  terrorist  groups  to  adapt  and  learn  are 
often  obvious  in  hindsight,  they  are  harder  to  identify  proactively.  If  we  are  going  to 
minimize  their  tactical  advantage,  however,  early  recognition  of  operational  growth  is 
critical.  Intelligence  analysts  need  to  be  able  to  extract  valuable  information  from  the 
“noisy  background  of  collected  intelligence,  and  the  analytical  tools  to  discern  what  the 
information  means  in  terms  of  terrorist  group  adaptive  activities.”i5*  In  order  to  make  a 
calculated  difference,  analysts  need  to  be  able  to  detect  terrorist  groups’  efforts  to  change 
and  adapt,  anticipate  whether  those  efforts  will  be  successful,  and  act  in  order  to  limit 
terrorist  groups’  ability  to  learn  or  undermine  their  learning  efforts. 

In  Aptitude  for  Destruction,  Brian  A.  Jackson  et  ah,  characterize  organizational 
learning  as  a  four-part  process  composed  of  acquiring,  interpreting,  distributing,  and 
storing  information  and  knowledge.!^*’  All  four  parts  must  exist  and  be  carried  out 
successfully  if  knowledge  is  to  become  organizational — that  is,  clearly  tied  to  group 
objectives,  accessible  to  many  different  group  members,  and  resistant  to  the  loss  of 
individual  members. This  approach  is  similar  to  a  corporate  strategy  where  the  system 
is  designed  to  survive  not  because  of  the  abilities  of  the  personnel  but  in  spite  of  them. 
The  successes  of  the  organization  are  procedurally  based  rather  than  personnel  based. 
This  is  evident  with  a  group  like  al-Qaeda  (AQ),  which  exists  in  2014  more  as  a 
philosophy  than  a  hierarchical  organization.  This  is  not  to  say  that  al-Qaeda  central 
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(AQC)  no  longer  operates,  but  rather  emphasize  its  many  splinter  organizations  thriving 
under  the  AQ  ideology.  1^2 


In  August  2001,  actor  James  Woods  saw  a  number  of  apparently  Middle  Eastern 
men  behaving  so  oddly  on  a  flight  from  New  York  to  San  Francisco  that  he  reported  them 
to  the  FBI.  This  was  possibly  a  test  run  for  what  would  later  be  the  September  1 1  high¬ 
jacking.  1^3  By  monitoring  routine  actions  by  a  transcontinental  flight  crew,  the  terrorists 
were  able  to  gather  significant  information  that  may  have  provided  the  operational 
groundwork  needed  to  carry  out  the  most  successful  terrorist  operation  in  history.  The 
knowledge  that  terrorists  conduct  “dry  runs”  offers  an  insight  into  their  meticulous 
planning  and  preparedness. 

1.  Tactical  Growth  of  Terrorist  Organizations 

Understanding  the  tactical  growth  of  two  specific  organizations  may  shed  some 
light  on  how  responders  can  react  to  an  increasing  threat  as  it  develops  rather  than 
reacting  to  the  aftermath  of  that  process. 

a.  The  Radical  Environmentalist  Movement 

Groups  such  as  the  Environmental  Fiberation  Front  (EFF)  and  the  Animal 
Fiberation  Front  (AFF)  have  existed  within  the  U.S.  as  leaderless  organizations  since  the 
early  nineties  fighting  for  the  rights  of  the  environment  and  animals  at  the  expense  of 
hard  working  Americans.  Since  1976,  1,100  criminal  acts  have  been  committed  in  the 
United  States  by  radical  environmentalist  groups,  resulting  in  more  than  $110  million  in 
property  damage  alone,  a  figure  that  does  not  include  the  significant  additional  costs 
associated  with  lost  research,  increased  security,  and  dampened  productivity.  1^4 

Groups  such  as  EFF  and  AFF  share  a  unique  organizational  structure  as  they  are 
essentially  leaderless.  The  EFF  considers  any  person  who  acts  on  behalf  of  the 


162  Margaret  Coker  and  Keith  Johnson,  “The  Death  of  Osama  Bin  Laden  As  A1  Qaeda  Adapts  Its 
Tactics,  Threat  From  Splinter  Groups  Persists,”  Wall  Street  Journal,  May  3,  201 1. 
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organization,  so  long  as  such  actions  are  consistent  with  ELF  guidelines,  to  be  an  “Elf,” 
or  agent  of  the  organization,  and  any  notions  undertaken  by  Elves  oan  be  considered 
actions  of  the  ELF  as  a  whole.  This  makes  it  difficult  to  track  organizational  learning 
and  knowledge  management  among  the  group. 

The  main  aim  is  to  base  a  group  on  so-oalled  phantom  oell  networks  or 
“autonomous  leadership  units”  (ALEls)  that  operate  completely 
independently  of  one  another,  but  whioh  are  able,  through  the  eombined 
force  of  their  aetions,  to  preeipitate  a  ehain  reaetion  that  eventually  leads 
to  a  national  revolution.  Integral  to  the  eoneept  is  information  teehnology 
(IT),  whioh  is  used  to  facilitate  oommunication  between  like-minded 
individuals  and  as  a  means  for  circulating  and  distributing  propaganda  and 
information.  Of  particular  importance  are  mediums  suoh  as  the  Internet, 
email  and  the  world  wide  web.  These  oyber  based  mediums  play  a  oruoial 
role  in  ensuring  that  all  individuals  are  kept  fully  abreast  of  events, 
allowing  for  planned  responses  and  attaoks  but  always  on  the  basis  of 
individual  initiative.  Used  in  this  way,  IT  both  overoomes  the  ‘tyranny  of 
distanoe’  and  obviates  the  neoessity  for  orders  and  direotives — thereby 
preoluding  the  need  or  a  physioal  group  [or  a  oentralized  leadership], 

b.  The  Provisional  Irish  Republican  Army 

The  Provisional  Irish  Republiean  Army  (PIRA)  has  a  30-year  history  of 
organizational  growth  both  strategically  and  tactically,  The  group  eonsistently  fought 
to  sustain  a  war  of  attrition  against  the  oeeupying  British  with  limited  suceess.^^^  One  of 
the  PIRA’s  alleged  leaders  summarized  the  group’s  strategy  as  “blattering  on  until  the 
Brits  leave.”  This  eaused  a  re-strueturing  of  organizational  goals. 
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“The  remarkable  evolution  of  this  organization  has  been  eharacterized  by  internal 
learning,  an  exeeptional  ability  to  adapt,  reorganize,  and  restrueture,  and  the  impressive 
development  of  a  highly  efficient  and  multi-dimensional  support  apparatus” 

While  the  PIRA  used  Rocket-Propelled  Grenades  (RPGs)  in  many  of  its  attacks, 
the  weapon’s  limitations  caused  the  group  to  develop  its  own  version,  the  Improvised 
Projected  Grenade  (IPGj.i’^i  The  newer  weapons  were  more  accurate  and  caused  less 
accidents  with  unintended  consequences;  accidents  which  damaged  sympathetic  support. 

As  the  PIRA  learned,  it  began  operations  aimed  at  specific  targets  which  furthered 
its  need  for  organizational  improvements.  As  it  increased  its  “out-of-theater”  targets  its 
intelligence  needs  also  increased.  This  lead  to  strategic  improvements  as  well.  “In  any 
conflict  the  tactical  efficiency  with  which  military  operations  are  executed  will  be 
meaningless  unless  they  form  part  of  a  coordinated  plan  to  achieve  political  ends,  because 
the  success  of  a  strategy  can  only  be  judged  with  reference  to  attainment  of  the  overall 

political  objective”  1^2 

c.  Summary  of  Organizational  Learning 

Both  the  PIRA  and  leaderless  environmental  groups  like  ELF  and  ALE  have 
benefitted  from  organizational  learning.  There  is  no  single  way  that  organizations  learn 
from  past  successes  and  failures.  ^23  Some  professionals  define  organizational  learning  as 
a  “change  in  behavior  in  response  to  stimulus,”  1^4  while  others  suggest  learning  is  sought 
after,  125  a  desire  to  improve  based  needs  and  shifting  objectives.  If  an  organization  is 
going  to  continue  to  survive,  it  will  have  to  benefit  from  a  combination  of  techniques  and 
frameworks.  Homeland  Security  professionals  will  have  to  follow  and  anticipate  the 
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organizational  development  of  terrorist  organizations  and  likewise,  have  to  develop  as 
organizations  themselves. 


C.  AGGRESSIVE  INTELLIGENCE 

Is  it  possible,  then,  to  use  exereise  design  as  an  offensive  weapon  more  so  than  a 
reactionary  support  tool.  Can  we,  by  combining  the  resources  of  an  intelligence  section 
with  that  of  our  exercise  design  capabilities,  seek  to  prepare  our  responders  and  civilians 
for  events  that  haven’t  happened  yet?  The  incorporation  of  competitive  intelligence  (Cl) 
into  a  cyclic  exercise  design  process  will  allow  an  agency  to  monitor  evolving  threats  that 
could  potentially  affect  their  agency  or  community  and  exercise  accordingly. 

While  the  idea  of  Cl  is  primarily  a  corporate  tactic,  it  allows  a  company  to 
anticipate  market  developments  proactively — rather  than  merely  react  to  them,  The 
company  is  more  able  to  remain  competitive  by  improving  its  strategic  decisions  and 
leading  to  better  performance  against  competitors.!^^  An  exercise  design  team,  charged 
with  the  responsibility  of  preparing  its  agency  for  unforeseen  disasters,  maintains  a 
similar  need  for  a  competitive  edge.  The  competitor,  in  this  case,  is  an  evolving  terrorist 
or  complex  natural  threat,  rather  than  a  rival  business.  Anticipating  the  learning  potential 
of  an  organization  will  help  “predict”  the  next  realistic  threat  and  attempt  to  learn  from 
that  prior  to  its  occurrence. 

D,  THE  MUMBAI— NAIROBI  CONNECTION 

The  2008  terrorist  attack  in  Mumbai  can  be  used  as  an  example  of  how 
intelligence  could  be  used  to  create  a  predictive  exercise.  In  November  of  2008,  10 
terrorists  attacked  several  locations  in  Mumbai,  India  killing  at  least  172  people.  Referred 
to  as  “India’s  9/1 1,”  the  attack  was  carried  out  inexpensively  and  without  the  use  of 
unconventional  weapons.  However,  parts  of  this  attack  were  noteworthy,  specifically,  its 
scope,  complexity  and  the  diversity  of  its  targets.  “The  prolonged  nature  of  the  episode. 
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which  went  on  for  60  hours  with  the  steadily  mounting  death  toll,  made  it  a  slow-motion 
shoot-out  and  siege  that  mesmerized  the  world’s  news  media.” 

Aceording  to  the  NYPD’s  analysis  of  the  attaek  at  the  Westgate  Mall  in  Nairobi, 
Kenya,  early  on  the  afternoon  of  September  21,  2013, 

four  terrorists  assoeiated  with  the  Somali  based  terrorist  organization  Al- 
Shabaab  launehed  an  armed  assault  using  rifles  and  hand  grenades  at  the 
Westgate  Shopping  Mall  in  Nairobi,  Kenya.  The  attaek  resulted  in  the 
deaths  of  67  people  and  it  took  almost  four  days  before  authorities 
deelared  the  seene  safe.  This  terrorist  attaek  was  the  largest  loss  of  life  in 
Kenya  due  to  terrorism  sinee  the  1998  bombing  of  the  U.S.  Embassy  in 
Nairobi  in  whieh  212  people  were  killed.  This  high  profile  armed  assault 
sueeessfully  raised  Al-Shabaab’s  worldwide  profile  as  a  terrorist 
organization.  It  also  elearly  illustrates  that  armed  assaults  by  terrorists  on 
“soft”  targets  sueh  as  a  shopping  mall  are  a  simple,  effeetive  and  easy  to 
eopy  taetie.i^^ 

Bruee  Hoffman  looked  at  Mumbai  as  a  benchmark  for  terrorist  suecess,  stating, 
“the  long,  drawn-out  assault  in  Mumbai  produeed  round  the  eloek  eoverage  around  the 
globe,  something  other  terrorist  groups  want  to  emulate.”!^*’  This  emulation  appears  to 
play  out  at  Westgate. 

To  summarize  the  two  events,  if  we  foeus  on  the  similarities  between  Mumbai 
and  Nairobi,  both  events  featured  moving  teams  of  attaekers  armed  with  assault  weapons 
and  grenades.  Both  targets  were  assumed  to  eontain  affluent  visitors  to  the  loeations  and 
it  eould  be  anticipated  that  most  of  the  vietims  would  likely  only  be  familiar  with  the  exit 
through  whieh  they  entered  minimizing  their  egress  options.  Both  events  lasted  for  days 
and  were  captured  live  for  the  world  to  see.  Terrorism’s  intent  is  to  intimidate  those 
watehing.  “Terrorism  is  meant  to  hurt,  not  to  destroy.’’^^^  use  of  gunfire,  as 
opposed  to  indiseriminate  use  of  bombs  as  used  previously  in  Mumbai  attacks  (2006 
Mumbai  train  attack,  in  which  209  people  were  killed  by  seven  bombs,  and  the  1993 


Angel  Rabasa  et  at,  The  Lessons  of  Mumbai  (Santa  Monica,  CA:  RAND,  2009). 

New  York  City  Police  Department,  Analysis  of  Al-Shabaab’s  Attack  at  the  Westgate  Mall  in 
Nairobi  Kenya,  New  York:  New  York  City  Police  Department,  November  1,  2013. 
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Mumbai  attack  where  257  persons  died  as  a  result  of  13  blasts  aeross  the  city),  provided 
the  “emotional  value”  sought  by  terrorists.  1^3 

There  are  similarities  in  the  Westgate  Mall.  If,  as  a  result  of  the  mall  shooting  in 
Nairobi  in  2013,  intelligence  analysts  discover  a  systemie  gap  in  evaeuation  polieies  in 
Nairobi,  then  that  analysis  will  be  shared  with  exereise  design.  A  joint  exereise  between 
fire,  emergency  medical  and  law  enforcement  personnel  as  well  as  private  loeal 
stakeholders  with  a  vested  interest  in  mass  evaeuation  of  customers  will  be  developed, 
conducted  and  evaluated.  The  loeation  is  not  as  important  as  the  similarities  in 
consumers.  For  example,  the  mall  in  Nairobi  eontained  a  transient  population,  marginally 
familiar  with  the  facility  and  not  fully  familiar  with  its  exits  or  available  areas  of  refuge. 
These  factors  influenced  the  ehaotie  movement  of  the  erowd  whieh  fueled  the  attaeker’s 
assault  targets. 

If  the  development  of  the  exercise  scenario  includes  multiple  insults  to  normal 
movement  with  various  points  of  egress  bloeked,  then  the  challenge  of  the  exercise 
beeomes  eompromised  evacuation.  The  evacuation  exercise  beeomes  the  benefit  to  the 
facility  as  a  result  of  the  intelligence  provided. 

1.  Lessons  Learned  from  Mumbai  and  Nairobi 

If  we  were  required  to  eonduet  an  after  aetion  review  of  the  two  events  from  the 
point  of  view  of  the  “bad  guys,”  what  would  be  areas  for  improvement?  Would  the 
following  recommendations  be  considered  appropriate? 

•  RECOMMENDATION:  While  the  fire  set  at  the  Taj  Hotel  was  effeetive 
in  garnering  media  coverage  for  a  prolonged  period,  it  should  have  been 
set  on  a  lower  floor  and  set  sooner.  would  ensure  more  trapped 

oeeupants  and  a  higher  eausality  count. 


1^3  Rabasa  et  al.,  The  Lessons  of  Mumbai,  6. 

Daniel  Howden  in  Nairobi,  “Terror  in  Westgate  Mall:  The  Full  Story  of  the  Attacks  That 
Devastated  Kenya,”  The  Guardian,  October  4,  2013. 

1^3  Rabasa  et  at.  The  Lessons  of  Mumbai. 


60 


•  RECOMMENDATION:  Disruptive  explosions  away  from  the  hotel 
eaused  confusion  among  responders.  Explosions  early  in  Nairobi  had  a 
similar  confusing  and  chaotic  effect. 

•  RECOMMENDATION:  Response  of  fire  and  police  units  is  often 
predictable  so  the  effectiveness  of  prepositioned  lEDs  should  slow 
response. 

•  RECOMMENDATION:  Use  of  a  “command  post”  allowed  for 
coordinated  communications  as  well  as  real  time  intelligence  for  attackers 
in  both  Mumbai  and  Nairobi. 

Another  incident  worthy  of  review,  at  this  point,  would  be  the  Westfield  Galleria 
Mall  arson  attack  form  October  of  2010.  During  this  Roseville,  California,  incident,  an 
armed  suspect,  believed  to  be  carrying  an  explosive  device,  entered  a  crowded,  upscale 
mall.  The  suspect  later  started  a  fire  and  barricaded  himself  in  the  back  of  one  of  the 
stores.  Through  the  confusion,  the  fire  sprinklers  were  deactivated  and  were  not 
reactivated  until  71  minutes  into  the  incident,  While  it  is  unclear  as  to  why  the 
sprinklers  were  deactivated,  what  is  clear  is  how  this  exacerbated  the  fire.  The  important 
lesson  learned  here,  for  the  would-be  terrorist  is  that  the  absence  of  the  fire  suppression 
system,  for  over  an  hour,  resulted  in  55  million  dollars  in  damage  to  the  mall.1^9  Qne  of 
the  goals  of  terrorism  is  financial  disruption  to  the  target.  The  motivation  of  the  attacker 
in  Roseville,  for  the  purposes  of  this  evaluation,  is  irrelevant.  What  is  relevant  is  that  one 
armed  attacker  occupied  a  mall,  terrorized  its  occupants  and  set  fire  to  the  facility, 
holding  responders  at  bay.  His  actions  caused  excessive  damage. 

E,  INTELLIGENT  EXERCISE  THEORY 

1.  Predictive  Scenario 

After  an  emergency,  such  a  focus  on  the  most  recent  event  is  common  and  the 
planners  often  concentrate  exclusively  on  planning  for  future  scenarios  of  the  same 
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type.i^*’  Ideally,  intelligence  analysts  should  focus  on  what  the  terrorists  view  as  their 
lessons  learned  and  provide  this  to  exercise  planners.  This  allows  for  the  development  of 
a  predictive  exercise  where  we  begin  to  prepare  for  what  may  be  next. 

If  terrorists  in  the  U.S.  were  intent  on  carrying  out  an  active  shooter  event,  then 
the  lessons  learned  from  Mumbai  and  Nairobi  should  help  in  the  planning  stage. 
Supplementing  this  intelligence  with  what  could,  conceivably,  be  learned  from  Roseville, 
should  yield  a  complex  active  shooter  event,  carried  out  by  four  to  six  gunmen  in  a 
reasonably  crowded  facility  e.g.,  a  mall,  theater  or  rail  station,  occupied  by  transient 
population  unfamiliar  with  all  the  exits  or  nuances  of  the  building. 

By  triggering  several  explosions  prior  to  entering  the  facility,  the  attackers  could 
cause  a  distraction.  Then,  by  compromising  the  fire  suppression  system  the  attackers 
could  begin  moving  through  the  facility  firing  their  weapons,  eventually  starting  the  fire 
and  either  trapping  victims  on  floors  above  or  siphoning  them  to  specific  exits  using  the 
directionality  of  the  fire.  This  corralling  would  leave  the  victims  extremely  vulnerable. 

If  this  event  were  to  take  place  in  a  location  with  an  extremely  vulnerable 
population,  a  hospital  for  example,  then  the  casualty  count  could  be  significant.  As  would 
the  emotional  impact  the  attack  would  have  on  a  U.S.  population.  A  New  York  hospital 
emergency  planner,  speaking  on  the  condition  of  anonymity,  stated  her  biggest  concern  in 
preparing  the  hospital  was  its  over  50  exits  and  the  hospital’s  inability  to  secure  them  all. 

The  incorporation  of  intelligence  allows  us  to  anticipate  what  challenges  may  lay 
ahead  and  begin  to  prepare  for  them. 

2,  Predictive  Exercise 

Taking  the  lessons  learned  from  previous  events  and  adding  them  to  what  we  can 
expect  would  be  an  attractive  soft  target  in  the  U.S.,  should  lead  us  to  conduct  evacuation 
exercises  that  focus  on  compromised  evacuation  routes.  As  people  unfamiliar  with 
additional  means  of  egress,  other  than  the  one  through  which  they  entered,  can  be 
expected  to  behave  as  part  of  a  crowd  during  unscheduled  evacuation,  we,  as  emergency 

190  Allan  McConnell  and  Lynn  Drennan,  “Mission  Impossible?  Planning  and  Preparing  for  Crisis,” 
Journal  of  Contingencies  and  Crisis  Management  14,  no.  2  (2006):  59-70. 
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planners,  need  to  provide  scenario  based  exercises  that  would  help  a  facility  develop 
alternative  response  plans  when  faced  with  a  chaotic,  aggressive  obstacle. 

When  developing  an  exercise  intended  to  prepare  people  for  any  emergency,  the 
scenario  is  not  nearly  as  important  as  are  the  objectives.  Using  the  intelligence  from 
above,  it  is  important  to  realize  that  our  exercises  do  not  need  to  mirror  our  intelligence, 
merely  shadow  the  objectives.  What  this  means  is  that  it  is  not  necessary  to  conduct  an 
active  shooter  exercise  in  a  hospital  to  provide  hospital  personnel  with  the  benefits  of  the 
intelligence  shared.  A  Tabletop  Exercise  (TTX)  conducted  in  a  hospital  with  discussions 
considering  a  variety  of  compromises  to  safety  should  lead  to  anticipatory  solutions. 

3.  Intelligent  Exercise  Cycle 
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Figure  11.  At  Left  is  the  Exercise  Cycle,  at  Right,  the  Intelligence  Cycle 

In  order  for  exercises  to  remain  current  and  relevant,  they  need  to  be  based  on 
new  intelligence.  Ideally,  this  will  provide  the  exercise  cycle  with  constant  input.  The 
cycle,  in  this  way,  however,  is  stagnant  within  the  exercise  agency.  As  the  new 
intelligence  is  fed  to  the  exercise  cycle,  the  exercises  are  conducted  and  evaluated, 
improvement  plans  are  developed  as  are  corrective  action  plans.  New  exercises  are 
created  and  the  cycle  continues.  As  the  intelligence  community  gains  new  information, 
this  is  analyzed  and  processed  then  disseminated  according  to  the  needs  of  the  consumer. 
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Feedback  is  received  from  within  the  intelligence  community  and  the  cycle  continues  as 
in  Figure.  These  two  cycles  can  benefit  from  each  other  through  their  mutual  cooperation. 
The  development  of  the  Intelligent  Exercise  Model  (lEM)  will  consist  of  integrated 
versions  of  both  the  exercise  design  cycle and  the  intelligence  cycle.  Each  cycle 
operates  independently  while  running  a  constant  flow  of  feedback  between  the  two 
entities.  As  evaluation  information  from  one  cycle  is  fed  into  the  other,  each  cycle 
supports  the  momentum  of  the  other,  theoretically  leaving  the  cycles  in  perpetual  motion 
dependent  upon  influences  from  each  other.  The  key  to  the  sustainability  of  the  procedure 
is  that  the  entire  process  is  constantly  fueled  by  new  intelligence.  The  feedback  from  the 
exercise  design  team  is  delivered,  post-exercise,  to  the  intelligence  producers.  They,  in 
turn,  continue  to  provide  intelligence  to  the  exercise  design  professionals  in  an  effort  to 
stay  informed  concerning  current  trends  and  threats. 
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Eigure  12.  Intelligent  Exercise  Model 


The  research  conducted  for  the  analysis  of  this  thesis  produced  the  Intelligent 
Exercise  Theory  (lET).  Which  can  be  defined  as  “The  use  of  intelligence  gained  from  a 


191”hsEEP-LLIS,”  Department  of  Homeland  Seeurity,  aecessed  February  6,  2014. 
https://www.llis.dhs.gov. 

192  Townsend  et  al.,  “Intelligence-Led  Mitigation,”  5. 

64 


disaster  or  terrorist  attack  to  prepare  for  future  events,  while  only  recreating  the  most 
salient  facets  of  the  event  to  drive  the  exercise  design  learning  cycle  in  anticipation  of 
future  crises.  This  is  to  suggest  that  reconstructing  the  entire  disaster  for  the  purposes  of 
exercise  is  not  nearly  as  effective  as  exercising  those  aspects  of  the  disaster  deemed 
significantly  influential  as  to  manipulate  reaction  to  future  disasters.  ” 

Exercises  are  often  location  driven.  This  means  that  if  a  facility  of  interest 
becomes  available,  the  calendar  turns  out  to  be  the  motivating  factor.  Projects  are  derived 
from  relationships  that  exist  between  agencies  and  a  mutual  interest  in  a  particular  facility 
rather  than  based  on  a  realistic  threat  assessment  of  locations.  This  limits  the  scope  of  the 
exercise.  By  narrowly  focusing  our  scenarios  based  on  the  availability  of  exercise 
partners,  rather  than  allowing  current  threats  and  vulnerabilities  to  steer  our  exercise 
goals  and  objectives,  we  are  limiting  opportunities  for  conducting  successful,  effective 
exercises.  The  union  of  intelligence  and  exercise  design  into  a  symbiotic  process  will 
simultaneously  broaden  the  inclusion  of  exercises  while  pinpointing  the  focus  of  design 
capabilities. 


Figure  13.  Technology  Acceptance  ModeE^s 


Since  the  lEM  integrates  variations  of  two  accepted  models;  the  intelligence 
cycle  and  the  HSEEPi^s  exercise  cycle,  participants  from  exercise  and  intelligence  are 
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already  familiar  with  them,  minimizing  the  learning  curve.  The  fact  that  these  two  models 
are  already  widely  accepted  means  that  there  should  already  exist  an  anticipated  ease  of 
use  as  well  as  an  expectation  of  value  added  to  the  process. 

It  is  important  to  understand  the  details  of  objectives.  We  can’t  find  a  solution 
until  we  have  understood  the  problem.  Gathering  information  from  many  sources  will 
broaden  the  field  of  information  and  eventually  improve  the  focus  of  the  project;  what  the 
final  project  is  supposed  to  look  like.  Initially,  an  increase  in  information  leads  to  better 
solutions,  but  there  comes  a  point  at  which  more  information  has  a  diminishing  effect. 
Information  by  itself  does  not  produce  solutions.  We  need  the  ability  to  work  with  the 
information  effectively,  The  incorporation  of  intelligence  into  exercise  design  provides 
intelligence  producers  and  analysts  with  a  measureable  consumer.  While  intelligence 
often  is  left  with  intangible  performance  successes,  the  consumer  and  producer  rarely 
exchange  thoughts.  1^7  xhe  development  of  exercises  based  directly  on  intelligence 
delivered  can  provide  a  performance  measure  and  apply  realistic  objectives,  as  well  as 
collaborative  feedback  based  on  new  intelligence  and  proposed  exercises.  The 
intelligence  officers  actually  contribute  and  participate  in  the  next  step  in  the  intelligence 
cycle. 

The  exercise  evaluation  will  then  result  in  a  comprehensive  After  Action  Report 
and  Improvement  Plan  (AAR/IP)  that  will  be  disseminated  among  responding  agencies 
and  the  private  stakeholders.  Developed  cooperatively  by  responders  and  private 
industry,  the  AAR/IP  can  benefit  all  agencies  involved  in  the  exercise.  Lessons  learned 
from  these  exercise  should  be  shared  among  participating  agencies  but,  more  importantly, 
made  available  to  non-participating  agencies  that  would  benefit  from  the  findings. 
Competitive  interests  between  competing  agencies  (i.e.,  hotels,  hospitals,  office 
buildings)  need  to  be  set  aside  and  allow  the  greater  interest  of  community  resilience  take 
precedence.  Each  individual  agency  or  stakeholder  needs  to  view  itself  first  as  a  member 


195  “HSEEP_Revision_Aprl3_Final,”  Department  of  Homeland  Security,  accessed  October  31,  2013, 
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of  the  community  and  secondly  as  an  individual  interest.  This  eliminates  multiple  in¬ 
groups  focused  on  their  own  needs  and  creates  a  collective  in-group  focused  on  the 
community’s  needs. 

This  is  the  idea  currently  in  use  in  Israel  where  the  community  is  focused  on 
the  protection  of  Israeli  interests  ahead  of  individual  interests.  The  Israeli  model  is  “based 
on  a  series  of  concentric  circles  of  increasing  protection,  with  the  individual  citizen  at  the 
center.”  199  xhis  is  a  definitive,  almost  literal  view  of  community  resilience,  where  the 
community  has  an  active  participatory  role  in  its  own  security.  To  provide  the  civilians 
with  the  necessary  tools  to  respond  appropriately  and  immediately  when  faced  with  an 
emergency,  the  Israel  Defense  Forces’  Home  Front  Command  (IDF  HFC)  orchestrates 
exercise  and  drills  that  prepare  the  civilian  population  to  interact,  where  necessary  with 
first  responders. 

Community  resilience  is  not  just  about  how  we  respond  during  times  of  crisis. 
Additionally,  it  raises  questions  that  challenge  how  we  build  strong  and  sustainable 
communities  in  the  face  of  the  demographic,  social,  and  economic  changes  of  the  21®' 
century.  For  example,  how  do  we  bring  together  government  and  private  partners 
equitably  in  planning  discussions?  How  do  communities  care  for  their  most 
vulnerable?  200  By  exercising  and  training  together,  each  agency  begins  to  understand  the 
capabilities  and  needs  of  the  other  in  an  effort  to  support  the  overall  vision  of  Israeli 
security.  This  is,  simply  stated,  the  tenet  of  community  resiliency.  20 1 

4.  New  Solutions  Create  New  Problems — Case  Study 

In  2008,  the  New  York  City  Fire  Department’s  (FDNY)  Exercise  Design  team 
created  a  bus  bombing  scenario  that  allowed  over  100  FDNY  responders  to  exercise  in  a 
realistic  atmosphere,  complete  with  an  eviscerated  bus,  numerous  simulated  injuries  and 
secondary  threats.  This  exercise,  consistent  with  global  events  and  threats,  i.e.,  the  2005 
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terrorist  attack  on  London’s  mass  transit  system,  was  later  incorporated  into  the  FDNY’s 
regular  training  schedule  and  allowed  every  member  of  the  Department,  fire  and  medical, 
to  participate.  In  May  of  2010,  members  of  the  FDNY  responded  to  an  attempted  car 
bomb  in  Times  Square  and  acted  in  accordance  with  their  training  to  safely  mitigate  a 
potential  disaster.  202  xhis  cycle,  consisting  of: 

Intelligence  Exercise  Design  Training  Practical  Application 

should  serve  as  the  model  for  Intelligence-led  Exercise  design.  The  events  of  May  2010 
actually  validate  the  lEM.  Consistent,  determined  training  provides  a  path  for  effective 
responder  performance. 

Any  new  procedure  runs  the  risk  of  failure.  To  incorporate  a  cooperative  working 
relationship  between  an  intelligence  section  and  its  agency’s  exercise  design  personnel  is 
going  to  run  somewhat  contradictory  to  current  operating  procedures.  Intelligence  is  now 
collected,  analyzed  and  evaluated  by  the  intelligence  section.  Intelligence  is  then 
packaged  for  dissemination.  The  lEM  will  need  at  least  a  year  to  test  for  effectiveness.  If 
ineffective,  the  transition  back  will  also  take  time.  This  could,  conceivably,  result  in  two 
year  absence  from  the  traditional  established  exercise  design  procedures  that  a  skilled 
team  has  grown  familiar  with.  If  affected  personnel,  reluctant  to  change  initially,  are  now 
faced  with  reinstituting  the  methods  they  had  fought  to  keep,  the  possibility  exists  of 
never  re-establishing  the  original  procedures. 

The  new  integration  may  also  develop  new  problems  not  yet  conceived.  In  order 
to  maintain  smooth  effectiveness,  these  will  have  to  be  identified  and  controlled  on  an 
individual  basis. 


202a1  Baker  and  Karin  Henry,  “Car  Bomb  Leads  to  Evacuation  in  Times  Square,”  New  York  Times, 
May  2,  2010. 
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VI.  CONCLUSION 


A.  SUMMARY  OF  ISSUES 

The  intent  of  this  thesis  was  to  explain  the  need  for  ineorporated  intelligence  into 
an  agency’s  exercise  design  model.  The  idea  of  exercise  design  is  to  validate  existing 
training.  The  addition  of  intelligence  into  the  planning  process  means  that  we  will  be 
preparing  to  meet  tomorrow’s  challenges  as  well  as  yesterdays.  This  allows  us  to  think 
forwardly  and  attempt  to  predict  where  our  future  challenges  lie,  rather  than  solving 
problems  where  we  have  already  demonstrated  measured  success. 

The  fusion  of  intelligence  and  exercise  design  will  produce  a  more  integrated 
exercise  product  that  the  FDNY  can  share  the  benefits  with  other  city  agencies  as  well  as 
private  sector  entities.  The  goal  is  not  simply  to  improve  the  exercise  design  methods  but 
to  increase  the  usability  of  exercise  design  as  a  building  block  of  community  resiliency. 
The  incorporation  of  intelligence  with  preparedness  efforts  is  essential  in  order  for  the 
community  to  achieve  true  resiliency.  Improving  public  and  private  communications  and 
the  exchange  of  information  is  possible  through  the  strengthening  of  these  relationships. 
The  new  efficient  development  of  exercise  design  should  highlight  exercise  deficiencies 
overlapping  the  concerns  of  city  agencies  and  private  industry  and  lead  to  the  creation  of 
exercises  that  produce  more  value  that  can  be  reinserted  into  the  exercise  design  model 
and  continue  to  build  on  the  cycle.  The  fusion  of  exercise  design  with  intelligence  will 
perpetuate  both  the  exercise  design  and  intelligence  cycles,  resulting  in  the  development 
of  predictive  exercises  that  build  a  closer  public-private  relationship  and  a  more  resilient 
community. 

The  plans  listed  above  include  a  recalibration  of  the  way  exercises  are  planned 
and  conducted;  from  a  stagnant,  scenario  driven  enterprise  to  one  fueled  by  current 
analytical  expertise. 

B,  FINAL  THOUGHTS 

If  we  wish  to  have  collaboration  between  agencies,  we  must  first  foster  an 

atmosphere  whereby  mutual  acceptance  of  ideas  exists.  By  using  intelligence  analyzed 
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from  real  world  events,  predictive  exercises  can  be  developed  incorporating  the  needs  of 
the  community,  public  and  private.  In  order  to  implement  this  strategy  to  benefit  a 
department  like  the  FDNY  and  a  city  like  New  York  will  require  the  creation  and  staffing 
of  additional  positions. 

Initially,  this  is  important  to  both  the  FDNY’s  Center  for  Terrorism  and  Disaster 
Preparedness’s  (CTDP)  Intelligence  division  and  Exercise  design  section.  The  success 
will  have  a  cascading  effect.  As  the  CTDP  conducts  more  intelligence  lead  exercises, 
those  involved  in  the  exercises,  the  FDNY  personnel  and  community  stakeholders,  will 
begin  to  enjoy  more  realistic  results  in  the  exercise.  This  can  be  measured  in  increased 
collaboration  in  future  exercises  as  well  as  increased  sharing  of  intelligence  between  the 
agency  and  private  stakeholders. 

The  idea  of  intelligence  driven  exercise  design  may  seem  a  simple  and  logical 
one.  We  receive  information  from  various  sources  every  day.  This  information  is 
analyzed  and  compared  to  what  is  already  known,  then  is  synthesized  and  distributed  as 
intelligence.  A  careful  analysis  of  this  intelligence  should  enable  us  to  develop  exercises 
that  look  forward  rather  than  backward,  to  allow  responders  to  learn  from  yesterday’s 
emergencies  and  prepare  for  tomorrow’s. 

C.  RECOMMENDATIONS 

While  the  core  of  this  thesis  was  directed  toward  the  exercise  design  and 
intelligence  communities,  the  recommendations  will  be  directed  primarily  toward  the 
FDNY  and  its  intelligence  and  exercise  needs.  In  order  to  successfully  implement  the 
intelligent  exercise  model,  the  following  recommendations  are  suggested. 

1,  Promote  Intelligent  Exercise  Theory 

As  this  thesis  has  made  clear.  Intelligent  Exercise  needs  to  be  the  future  of 
exercise;  and  the  future  of  intelligence.  The  two  communities  need  to  realize  their 
significance  to  one  another  and  work  on  developing  mutual  objectives.  As  each  realizes 
the  other  is  part  of  their  in-group  and  that  they  are  not  individual  entities  competing  for 
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the  same  eommodity  but  partners  working  toward  the  same  goals,  they  ean  begin  to 
develop  intelligent  exercises  that  benefit  the  entire  community. 

2,  Clearly  Define  the  Exercise  Design  Model 

Intelligent  exercise  is  a  simple  fusion  of  the  intelligence  cycle  and  the  HSEEP 
exercise  design  cycle,  the  process  needs  to  be  fully  explained.  The  benefits  that  come 
from  the  intersection  of  the  cycles  at  the  evaluation  process  drive  the  incorporation  of 
intelligence  into  exercise.  The  re-evaluation  process  will  have  to  be  apparent  as  will  an 
anticipated  benefit  and  an  obvious  ease  of  use. 

3.  Create  an  Exercise  Intelligence  Liaison  Officer 

This  position  would  serve  as  a  liaison  between  exercise  design  and  intelligence. 
Not  dedicated  to  either  exercise  design  or  intelligence,  but  committed  to  the  fusion  of 
both  interests,  the  liaison  position  will  function  as  a  bridge  between  current  trends  and 
future  exercises.  By  understanding  the  principals  of  intelligence  gathering  and 
information  sharing  as  well  as  the  fundamentals  of  exercise  design,  the  officer  assigned  to 
this  task  can  link  the  two  fields  in  an  effort  to  increase  the  value  of  preparedness 
exercises.  It  would  also  serve  the  fusion  of  the  two  sections  to  obtain  a  level  of  security 
clearance  for  this  officer  so  he  may  further  bridge  the  gap  between  the  fields.  The  cost 
will  be  dependent  on  the  member  filling  the  new  position.  This  can  be  satisfied  by 
assigning  a  current  member  of  either  the  intelligence  section  familiar  with  exercise  design 
or  similarly  with  a  member  of  the  exercise  design  team  possessing  intelligence 
experience.  Since  both  sections  are  already  established,  the  impact  on  cost  should  be 
minimal.  The  cost  can  be  split  across  the  budgets  already  allocated  for  the  two  sections. 
However,  transitioning  a  member  between  both  sections  will  essentially  leave  both 
sections  with  a  partial  personnel  deficit  as  the  new  position  will  remove  them  from 
current  responsibilities.  Conversely,  if  a  new  member  needs  to  be  assigned  from  outside 
the  Center,  the  cost  will  be  consistent  with  the  member’s  pay  rate  and  will  require  new 
budget  lines.  The  creation  of  a  new  position  will  definitely  have  an  impact  on  the  grant 
funded  budget  that  the  CTDP  survives  on. 
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This  initial  financial  infiuence  can  be  offset  by  future  antieipated  effieieney.  By 
inereasing  the  value  of  exereises  eondueted  and  relying  on  partieipation  from  private 
stakeholders,  the  quantity  of  exercises  eondueted  should  deerease  while  the  value  per 
exereise  inereases.  It  ean  be  antieipated  that  the  cost  of  fulfilling  the  new  Liaison  position 
will  be,  at  least  partially,  eompensated  through  the  efficiency  of  the  new  proeedure. 

The  implementation  proeedure  should  not  be  lengthy  but  identifying  the  new 
exereise  intelligenee  liaison  offieer  may  take  several  months.  The  position  is  new  and 
untested.  Identifying  an  appropriate  offieer  who  is  a  Master  Exereise  Praetitioner  (MEP) 
and  an  experieneed  and  qualified  Intelligenee  analyst  will  be  time  consuming.  A  person 
with  these  speeific  characteristics  may  not  exist  and  the  position  may  be  filled  by  an 
individual  with  only  partial  qualifieations.  Also,  onee  filled,  as  the  position  is  un¬ 
established,  there  will  be  a  positional  learning  eurve.  The  size  of  the  learning  eurve  will 
be  dependent  on  individual  skills. 

4.  Develop  a  N.Y.  City-Centric  Exercise  Design  Platform 

Similar  to  the  DHS  National  Eessons  Teamed  Information  Sharing  (EEIS),203  this 
online  platform  would  actively  seek  collaboration  citywide  when  intelligenee  dictates  the 
need  for  a  speeifie  exercise.  Made  up  of  publie  and  private  stakeholders,  ageneies  would 
be  responsible  to  share  intelligenee  and  ideas  and  offer  personnel  committed  to  the  design 
team.  At  the  eonelusion  of  the  exereise,  eollaboration  on  the  AAR  would  result  in  a 
shared  NYC  Eessons  Teamed  (NYCEE)  platform  that  would  serve  as  a  foundation  for 
future  eollaborative  exereises.  Transfer  of  knowledge  is  not  an  easily  identified 
proeess,204  but  the  potential  return  on  investment  that  ean  be  assoeiated  with  eollaborated 
exereises  ean  be  assumed  to  be  signifieant.  When  multiple  ageneies  look  at  an  issue 
through  a  variety  of  perspeetives  we  ean  gain  greater  insight  into  potential  areas  of 
eoneem  invisible  to  one  ageney  or  another. 

As  a  result  of  its  commitment  to  improvement  through  exereise,  the  FDNY 
eurrently  employs  over  40  Master  Exereise  Praetitioners  (MEP).  Many  NY  eity  ageneies 

203  “HSEEP-LLIS,”  https://www.llis.dhs.gov/. 

204  Eriksson,  “Knowledge  Transfer, “  162-16. 
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have  zero.  That  doesn’t  mean  they  do  not  have  an  interest  in  exereise,  they  may  not  have 
the  personnel  or  resourees  available.  This  limits  their  eapability  to  eontribute.  By  making 
lessons  learned  and  AARs  available  on  an  open  souree,  citywide  platform,  each  agency 
would  have  the  potential  to  learn  from  every  exercise  and  all  city  agencies. 

As  a  result  of  the  NYCLL,  agencies  without  exercise  design  personnel  could  also 
find  more  openness  in  the  NYC  exercise  design  community  allowing  them  to  observe 
and/or  contribute  to  exercises  from  other  agencies  and  remain  within  their  budgetary  or 
personnel  constraints. 

New  ideas  drive  change.  Change,  however,  is  often  met  with  resistance.  This 
resistance  is  what  we  need  to  overcome.  The  idea  of  moving  procedures  forward,  while 
on  the  surface  would  seem  to  resonate  progress,  is  frightening  or  intimidating  to  some. 
Likewise,  interagency  biases  may  prevent  collaboration  between  agencies,  public  and 
private.  While  many  agencies  are  comfortable  exposing  deficiencies  internally,  they  tend 
to  be  less  comfortable  having  these  publically  known.  xhis  can  be  true  internally  as 
well  as  externally.  If  each  section  views  itself  as  part  of  the  in-group  and  the  other  section 
as  the  out-group,  then  collaboration  will  not  flourish. 206  By  developing  exercises  based 
on  intelligence  together,  both  sections  will  realize  that  they  are  part  of  the  same  in-group. 
Only  then  can  true  effective  collaboration  be  expected  to  grow. 

D,  METRICS  OF  SUCCESS 

Stakeholders  will  look  to  participate  in  the  development  of  these  exercises,  as 
FDNY  exercise  design  improves  its  reputation  among  the  city  exercise  design 
community.  Insurance  underwriters,  aware  of  the  benefits  reaped  by  their  clients  will 
encourage  this  participation  by  offering  incentives  to  these  stakeholders.  Therefore,  the 
stakeholders,  private  industry,  can  offset  the  cost  of  the  exercise  by  partially  funding  this 
exercise  based  on  the  cost  savings  rewarded  them  by  their  insurance  companies. 
Consequently,  by  improving  the  exercise  design  product  and  exploiting  its  collaborative 
benefits  citywide,  the  new  process  can  become  self  funding. 

205  Best,  Intelligence  Information. 

206  Tajfel,  Social  Identity  and  Intergroup  Relations. 
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In  order  to  determine  successfulness,  progress  will  have  to  be  measured 
incrementally.  First,  the  length  of  time  to  identify  an  Intelligence  exercise  liaison  should 
serve  as  a  significant  indicator  of  anticipated  success.  A  prolonged  search  with  several 
starts  and  stops  could  be  indicative  of  future  systemic  problems.  Conversely,  if  the 
position  is  filled  quickly  and  the  transition  begins  cooperatively,  expectations  of  success 
are  realistic.  The  positional  requirements  are  only  part  of  the  anticipated  systemic 
concerns. 

The  true  measure  of  success  will  be  the  observation  of  increased  collaborative 
interest  among  additional  city  agencies  (e.g.  NYPD)  and  private  stakeholders  with  FDNY 
exercise  design.  It  is  possible  that  knowledge  developed  from  the  analytic  preparedness 
activities  may  be  transferred  to  individuals  that  participated  in  the  exercises  by 
discussions  or  other  activities  not  necessarily  acknowledged  by  the  individuals. This 
means  that  some  of  the  benefits  derived  during  the  exercises  may  simply  result  in 
performance  improvements  with  individual  applications.  Employees  and  responders  may 
find  themselves  more  prepared  and  familiar  with  their  evacuation  and  response 
capabilities  in  ways  that  are  not  tangibly  measureable,  but  result  in  overall  improvements 
that  extend  to  non-emergency  situations. 

E,  POTENTIAL  OPPOSITION 

There  is  an  organizational  reluctance  to  change  for  several  reasons.  Teemu 
Reiman,  and  Carl  Rollenhagen  outline  human  and  organizational  biases  that  have  an 
effect  on  the  management  of  safety  in  four  thematic  areas:  beliefs  about  human  behavior, 
beliefs  about  organizations,  beliefs  about  information  and  safety  models. 208  it  is  possible 
that  the  new  procedure  will  be  met  with  resistance  from  both  the  exercise  design  and 
intelligence  sections.  The  fusion  of  exercise  and  intelligence  is  not  the  way  things  are 
currently  done  and  siloed  operations  are  often  preferred  to  change.  The  two  divisions 
currently  operate  with  stovepipe  functionality.  That  is  to  say,  while  they  are  certainly 
aware  of  each  other,  there  exists  a  territorial  protection  around  which  each  section 


202  Eriksson,  “Knowledge  Transfer,”  162-169 

208  Reiman  and  Rollenhagen,  “Human  and  Organizational  Biases.” 
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operates  and  cooperative  analysis  rarely  occurs.  The  inclusion  of  intelligence  into 
exercise  may  result  in  mission  creep,  where  exercise  design  feels  that  intelligence  is 
guiding  their  exercise  plans.  Likewise,  Intelligence  may  feel  as  if  Exercise  is  trying  to 
coax  intelligence  to  fit  the  desired  exercise  plan.  The  two  sections  will  have  to  realize  a 
common  goal  and  work  together  to  achieve  it.  Each  will  also  have  to  respect  the  role  of 
the  other  in  order  to  maintain  unified  objectives.  In  other  words,  each  section  will 
function  as  independent  elements  of  the  same  corporation  realizing  the  same  goals.  In 
order  to  achieve  this,  an  exercise  can  be  developed  with  the  interests  of  the  intelligence 
section  featured  prominently.  By  including  the  intelligence  personnel  in  each  step  of  the 
exercise  process,  they  can  see,  firsthand,  the  benefits  that  will  be  gleaned  from  this 
fusion.  Allowing  them  to  have  structural  input  toward  the  completed  exercise  will  let 
them  direct  the  exercise  to  include  the  intelligence  they  have  analyzed  including 
anticipated  actions  by  players  and  antagonists,  whether  man-made  or  natural. 

Intelligence  has  historically  been  labeled  as  a  trade  that  shares  information 
reluctantly.  209  Exercise  design  needs  to  establish  itself  as  a  legitimate  consumer  of 
intelligence  in  order  to  encourage  intelligence  sharing.  To  accomplish  this,  exercise 
design  is  going  to  have  to  demonstrate  a  value  added  to  the  intelligence  community.  The 
incorporation  of  the  two  cycles  demonstrates  this.  The  evaluation  step  in  each  cycle  is 
designed  to  support  the  movement  of  the  other.  Exercise  design  can  provide  valuable 
action  items  recovered  after  an  exercise  showing  a  need  for  additional  intelligence. 

After  Action  Reports  written  at  the  conclusion  of  an  exercise  can  show  successes 
from  intelligence  inserted  into  the  process  but,  more  importantly,  exhibit  a  need  for 
additional  intelligence. 

This  model  is  asking  the  EDNY  intelligence  community  to  share  openly.  Not  only 
to  share  intelligence  but  to  offer  analysis  relevant  to  the  exercise  design  community.  The 
fact  that  they  are  being  asked  to  share  internally  may  not  matter.  This  transition  from  two 
independent  sections  to  two  parts  of  the  same  whole  operating  cohesively  may  meet  some 
opposition. 


209  National  Commission  on  Terrorist  Attacks  upon  the  United  States,  The  9/11  Commission  Report. 
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Specialized  training  is  required  to  become  an  Intelligence  analyst.  Exercise  design 
professionals  receive  training  to  become  Master  Exercise  Practitioners  (MEPs).  Both 
specialists  are  proud  of  their  certifications  and  protective  of  them.  If  the  model  is 
successfully  implemented  within  the  FDNY,  then  it  could  serve  as  the  ideal  for 
incorporation  between  the  two  communities  beyond  the  Department.  This  could  create 
territorial  defensiveness  by  both  communities,  if  either  the  intelligence  or  exercise  design 
community  feels  the  other  is  attempting  to  assume  their  responsibilities  without  the 
benefit  of  appropriate  training. 

F.  ADDITIONAL  RESEARCH  NEEDED 

Several  areas  require  additional  research  in  the  area  of  Intelligent  Exercise.  The 
relationship  between  recommendations  and  objectives  needs  to  be  expanded.  While  the 
research  conducted  was  sufficient  for  this  study,  it  should  be  continually  expanded  and 
monitored.  This  will  produce  evidence  of  improvement  trends  or  show  continued 
stagnation  in  areas  that  require  improvements. 

The  Intelligent  Exercise  Theory  and  the  Intelligent  Exercise  Model  need  to  be 
exercised.  A  viable  metric  must  be  developed  by  which  the  theory  can  be  tested  and 
evaluated  and  resources  and  time  committed  to  its  use  and  necessary  improvement.  To 
simply  adapt  a  new  policy  without  a  willingness  to  support  its  progress  is 
counterproductive  to  the  theory’s  premise. 

Collaboration  should  be  part  of  every  agency’s  strategic  plan.  Public  and  private 
agencies  must  perceive  themselves  as  part  of  the  larger  community  first  and  foremost  and 
realize  that  their  participation  in  an  agency  should  come  secondary.  Community 
resiliency  should  be  the  goal  of  every  member  of  the  community. 
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SUPPLEMENTAL 


Analysis  of  101  After  Action  Reports  Research  conducted  for  this  thesis  was 
based  on  101  After  Action  Reports  (AAR)  that  were  evaluated  for  themes  in 
recommendations  and  objectives.  The  AARs  were  separated  by  real-world  events  and 
exercises.  Each  exercise  AAR  was  coded  for  inclusion  into  71  categories  of  objectives 
and  107  categories  of  recommendations.  Each  real-world  AAR  was  coded  for 
recommendations  in  the  same  categories.  The  frequencies  of  occurrence  were  then 
compared  and  analyzed. 

Those  interested  in  obtaining  the  supplemental  can  contact  Dudley  Knox  Eibrary 
at  Naval  Postgraduate  School. 
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